Emotional resonance scoring of a visual stimulus

ABSTRACT

An apparatus includes a processing device configured to obtain an information density matrix characterizing information density of feature points in an input visual stimulus, and to analyze content of the input visual stimulus utilizing the information density matrix to identify one or more objects in the input visual stimulus. The processing device is also configured to determine emotional resonance scores associated with each of the objects in the input visual stimulus, a given emotional resonance score for a given one of the one or more objects characterizing an intensity and an emotional valence of an emotional effect of the given object on a user viewing the input visual stimulus. The processing device is further configured to modify a design of the input visual stimulus to adjust at least one of the emotional resonance scores associated with at least one of the one or more objects in the input visual stimulus.

FIELD

The field relates generally to information processing, and moreparticularly to managing content in information processing systems.

BACKGROUND

Various organizations develop designs that include a visual stimuluswith various information content that an organization is seeking tocommunicate via that visual stimulus. An organization, for example, mayprovide users with multiple products (e.g., applications, websites andwebpages, etc.) for presenting a given design. It can be difficult,however, to adapt the given design for different computing devices,output formats, for individual differences across viewers of the givendesign, etc. It is also difficult to quantitatively measure theeffectiveness of the given design, such as to ensure that the givendesign communicates a desired amount and type of information.

SUMMARY

Illustrative embodiments of the present disclosure provide techniquesfor emotional resonance scoring of a visual stimulus.

In one embodiment, an apparatus comprises at least one processing devicecomprising a processor coupled to a memory. The at least one processingdevice is configured to perform steps of obtaining an informationdensity matrix for an input visual stimulus, the information densitymatrix characterizing information density of feature points in the inputvisual stimulus, and analyzing content of the input visual stimulusutilizing the information density matrix to identify one or more objectsin the input visual stimulus. The at least one processing device is alsoconfigured to perform the step of determining emotional resonance scoresassociated with each of the one or more objects in the input visualstimulus, a given emotional resonance score for a given one of the oneor more objects characterizing an intensity and an emotional valence ofan emotional effect of the given object on a user viewing the inputvisual stimulus. The at least one processing device is furtherconfigured to perform the step of modifying a design of the input visualstimulus to adjust at least one of the emotional resonance scoresassociated with at least one of the one or more objects in the inputvisual stimulus.

These and other illustrative embodiments include, without limitation,methods, apparatus, networks, systems and processor-readable storagemedia.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an information processing system includingan information density toolkit system in an illustrative embodiment.

FIGS. 2A-2I show a system flow for an information density mapping toolin an illustrative embodiment.

FIGS. 3A and 3B show examples of heatmap overlays for two input designimages along with associated cognitive load index values in anillustrative embodiment.

FIG. 4 shows another example of a heatmap overlap for an input imagealong with a cognitive load index in an illustrative embodiment.

FIG. 5 is a flow diagram of an exemplary process for information densitymapping of visual stimulus in an illustrative embodiment.

FIG. 6 is a table showing scenarios and their interpretation for spatialclustering of feature values in an illustrative embodiment.

FIG. 7 is a graph illustrating ranges of visual complexity in anillustrative embodiment.

FIGS. 8A and 8B show an example application of information density heatmapping and automated cluster analysis for an input product page in anillustrative embodiment.

FIGS. 9A and 9B show another example application of information densityheat mapping and automated cluster analysis for an input product page inan illustrative embodiment.

FIG. 10 shows a plot of a generalized linear model indicating thatcognitive load index is a significant predictor of conversion rates inan illustrative embodiment.

FIG. 11 is a flow diagram of an exemplary process for cognitive loadscoring of a visual stimulus in an illustrative embodiment.

FIG. 12 shows sets of possible behavioral intents associated with a usertraversing a set of webpages in an illustrative embodiment.

FIG. 13 shows a system flow for a behavioral intent estimation tool inan illustrative embodiment.

FIG. 14 is a flow diagram of an exemplary process for behavioral intentestimation for a visual stimulus in an illustrative embodiment.

FIG. 15 shows a system flow for a color-emotion analysis tool in anillustrative embodiment.

FIG. 16 is a flow diagram of an exemplary process for color-emotionanalysis of a visual stimulus in an illustrative embodiment.

FIG. 17 shows a system flow for an emotional resonance scoring tool inan illustrative embodiment.

FIG. 18 shows an example of generation of emotional resonance scores foran input image in an illustrative embodiment.

FIG. 19 is a flow diagram of an exemplary process for emotionalresonance scoring of a visual stimulus in an illustrative embodiment.

FIGS. 20A and 20B show examples of eye-tracking fixation patterns in anillustrative embodiment.

FIG. 21 shows a plot illustrating correlation between fixation time andlikelihood of purchasing a product in an illustrative embodiment.

FIG. 22 shows a system flow for a visual attention likelihood estimationtool in an illustrative embodiment.

FIG. 23 shows an example of generation of visual attention likelihoodestimates in an illustrative embodiment.

FIG. 24 is a flow diagram of an exemplary process for visual attentionlikelihood estimation of a visual stimulus in an illustrativeembodiment.

FIGS. 25 and 26 show examples of processing platforms that may beutilized to implement at least a portion of an information processingsystem in illustrative embodiments.

DETAILED DESCRIPTION

Illustrative embodiments will be described herein with reference toexemplary information processing systems and associated computers,servers, storage devices and other processing devices. It is to beappreciated, however, that embodiments are not restricted to use withthe particular illustrative system and device configurations shown.Accordingly, the term “information processing system” as used herein isintended to be broadly construed, so as to encompass, for example,processing systems comprising cloud computing and storage systems, aswell as other types of processing systems comprising variouscombinations of physical and virtual processing resources. Aninformation processing system may therefore comprise, for example, atleast one data center or other type of cloud-based system that includesone or more clouds hosting tenants that access cloud resources.

FIG. 1 shows an information processing system 100 configured inaccordance with an illustrative embodiment. The information processingsystem 100 includes a set of client devices 102-1, 102-2, . . . 102-M(collectively, client devices 102) which are coupled to a network 104.Also coupled to the network 104 is an IT infrastructure 105 comprisingone or more IT assets 106, an information database 108, and aninformation density toolkit system 110. As used herein, “informationdensity” refers to a measure of the amount and compactness of visualfeatures within a design (e.g., a digital design, such as a webpage, awebpage component, a scrollable image, document, etc.). The IT assets106 may comprise physical and/or virtual computing resources in the ITinfrastructure 105. Physical computing resources may include physicalhardware such as servers, storage systems, networking equipment,Internet of Things (IoT) devices, other types of processing andcomputing devices including desktops, laptops, tablets, smartphones,etc. Virtual computing resources may include virtual machines (VMs),containers, etc.

The IT assets 106 of the IT infrastructure 105 may host applicationsthat are utilized by respective ones of the client devices 102, such asin accordance with a client-server computer program architecture. Insome embodiments, the applications comprise web applications designedfor delivery from assets in the IT infrastructure 105 to users (e.g., ofclient devices 102) over the network 104. Various other examples arepossible, such as where one or more applications are used internal tothe IT infrastructure 105 and not exposed to the client devices 102. Itshould be appreciated that, in some embodiments, some of the IT assets106 of the IT infrastructure 105 may themselves be viewed asapplications that are to be analyzed for information density.

The information density toolkit system 110 utilizes various informationstored in the information database 108 in analyzing input (e.g., sourceimages). Such analysis may include, but is not limited to, determiningthe information density of the input, a cognitive load (e.g., theinformation processing demands placed on an individual's brain whilecompleting a task) of the input, behavioral intent of users viewing theinput, color-emotion analysis of the input, emotional resonance of theinput, visual attention likelihood for different portions of the input,etc. In some embodiments, the information density toolkit system 110 isused for or by an enterprise system. For example, an enterprise maysubscribe to or otherwise utilize the information density toolkit system110 for analyzing input (e.g., source documents, webpages, applications,etc. which are generated and made available or accessed via the ITassets 106 of the IT infrastructure 105, on client devices 102 operatedby users of the enterprise, etc.). As used herein, the term “enterprisesystem” is intended to be construed broadly to include any group ofsystems or other computing devices. For example, the IT assets 106 ofthe IT infrastructure 105 may provide a portion of one or moreenterprise systems. A given enterprise system may also or alternativelyinclude one or more of the client devices 102. In some embodiments, anenterprise system includes one or more data centers, cloudinfrastructure comprising one or more clouds, etc. A given enterprisesystem, such as cloud infrastructure, may host assets that areassociated with multiple enterprises (e.g., two or more differentbusiness, organizations or other entities).

The client devices 102 may comprise, for example, physical computingdevices such as IoT devices, mobile telephones, laptop computers, tabletcomputers, desktop computers or other types of devices utilized bymembers of an enterprise, in any combination. Such devices are examplesof what are more generally referred to herein as “processing devices.”Some of these processing devices are also generally referred to hereinas “computers.” The client devices 102 may also or alternately comprisevirtualized computing resources, such as VMs, containers, etc.

The client devices 102 in some embodiments comprise respective computersassociated with a particular company, organization or other enterprise.Thus, the client devices 102 may be considered examples of assets of anenterprise system. In addition, at least portions of the informationprocessing system 100 may also be referred to herein as collectivelycomprising one or more “enterprises.” Numerous other operating scenariosinvolving a wide variety of different types and arrangements ofprocessing nodes are possible, as will be appreciated by those skilledin the art.

The network 104 is assumed to comprise a global computer network such asthe Internet, although other types of networks can be part of thenetwork 104, including a wide area network (WAN), a local area network(LAN), a satellite network, a telephone or cable network, a cellularnetwork, a wireless network such as a WiFi or WiMAX network, or variousportions or combinations of these and other types of networks.

The information database 108, as discussed above, is configured to storeand record various information that is used by the information densitytoolkit system 110. Such information may include, but is not limited to,information that is to be analyzed (e.g., for information density,cognitive load, behavioral intent, color-emotion analysis, emotionalresonance, and visual attention likelihood). In some embodiments, theinformation database 108 is a source of input (e.g., one or more images)that are to be analyzed by the information density toolkit system 110.The information database 108 may therefore represent one or more datasources. The information database 108 may also or alternatively storethe output of analysis of such input by the information density toolkitsystem 110. The information database 108 in some embodiments isimplemented using one or more storage systems or devices associated withthe information density toolkit system 110. In some embodiments, one ormore of the storage systems utilized to implement the informationdatabase 108 comprises a scale-out all-flash content addressable storagearray or other type of storage array.

The term “storage system” as used herein is therefore intended to bebroadly construed, and should not be viewed as being limited to contentaddressable storage systems or flash-based storage systems. A givenstorage system as the term is broadly used herein can comprise, forexample, network-attached storage (NAS), storage area networks (SANs),direct-attached storage (DAS) and distributed DAS, as well ascombinations of these and other storage types, includingsoftware-defined storage.

Other particular types of storage products that can be used inimplementing storage systems in illustrative embodiments includeall-flash and hybrid flash storage arrays, software-defined storageproducts, cloud storage products, object-based storage products, andscale-out NAS clusters. Combinations of multiple ones of these and otherstorage products can also be used in implementing a given storage systemin an illustrative embodiment.

Although not explicitly shown in FIG. 1 , one or more input-outputdevices such as keyboards, displays or other types of input-outputdevices may be used to support one or more user interfaces to theinformation density toolkit system 110, as well as to supportcommunication between the information density toolkit system 110 andother related systems and devices not explicitly shown.

The client devices 102 are configured to access or otherwise utilize theIT infrastructure 105. In some embodiments, the client devices 102 areassumed to be associated with system administrators, IT managers orother authorized personnel responsible for managing the IT assets 106 ofthe IT infrastructure 105 (e.g., where such management includesperforming analysis of input that is generated and made available by, oraccessed by the IT assets 106, or of applications or other software thatruns on the IT assets 106). For example, a given one of the clientdevices 102 may be operated by a user to access a graphical userinterface (GUI) provided by the information density toolkit system 110to analyze input (e.g., one or more source images). The informationdensity toolkit system 110 may be provided as a cloud service that isaccessible by the given client device 102 to allow the user thereof toanalyze some input. In some embodiments, the IT assets 106 of the ITinfrastructure 105 are owned or operated by the same enterprise thatoperates the information density toolkit system 110 (e.g., where anenterprise such as a business provides support for the assets itoperates). In other embodiments, the IT assets 106 of the ITinfrastructure 105 may be owned or operated by one or more enterprisesdifferent than the enterprise which operates the information densitytoolkit system 110. Various other examples are possible.

In some embodiments, the client devices 102 may implement host agentsthat are configured for automated transmission of information to andfrom the information density toolkit system 110. Such host agents mayalso or alternatively be configured to automatically receive from theinformation density toolkit system 110 commands to execute remoteactions (e.g., to modify input or source images that are generated andmade available to or accessed on or by the client devices 102 and/or theIT assets 106 of the IT infrastructure 105). Host agents may similarlybe deployed on the IT assets 106 of the IT infrastructure 105. It shouldbe noted that a “host agent” as this term is generally used herein maycomprise an automated entity, such as a software entity running on aprocessing device. Accordingly, a host agent need not be a human entity.

The information density toolkit system 110 in the FIG. 1 embodiment isassumed to be implemented using at least one processing device. Eachsuch processing device generally comprises at least one processor and anassociated memory, and implements one or more tools (e.g., functionalfeatures or logic) for controlling certain features of the informationdensity toolkit system 110. In the FIG. 1 embodiment, the informationdensity toolkit system 110 comprises an information density mapping tool112, a cognitive load index generation tool 114, a behavioral intentestimation tool 116, a color-emotion analysis tool 118, an emotionalresonance scoring tool 120, a visual attention likelihood estimationtool 122, and a design personalization tool 124. Functionality of thevarious tools of the information density toolkit system 110 will bedescribed in further detail below.

It is to be appreciated that the particular arrangement of the clientdevices 102, the IT infrastructure 105, the information database 108 andthe information density toolkit system 110 illustrated in the FIG. 1embodiment is presented by way of example only, and alternativearrangements can be used in other embodiments. As discussed above, forexample, the information density toolkit system 110 (or portions ofcomponents thereof, such as one or more of the information densitymapping tool 112, the cognitive load index generation tool 114, thebehavioral intent estimation tool 116, the color-emotion analysis tool118, the emotional resonance scoring tool 120, the visual attentionlikelihood estimation tool 122, and the design personalization tool 124may in some embodiments be implemented internal to one or more of theclient devices 102 and/or the IT infrastructure 105.

At least portions of the information density mapping tool 112, thecognitive load index generation tool 114, the behavioral intentestimation tool 116, the color-emotion analysis tool 118, the emotionalresonance scoring tool 120, the visual attention likelihood estimationtool 122, and the design personalization tool 124 may be implemented atleast in part in the form of software that is stored in memory andexecuted by a processor.

The information density toolkit system 110 and other portions of theinformation processing system 100, as will be described in furtherdetail below, may be part of cloud infrastructure.

The information density toolkit system 110 and other components of theinformation processing system 100 in the FIG. 1 embodiment are assumedto be implemented using at least one processing platform comprising oneor more processing devices each having a processor coupled to a memory.Such processing devices can illustratively include particulararrangements of compute, storage and network resources.

The client devices 102, the IT infrastructure 105, the informationdatabase 108 and the information density toolkit system 110 orcomponents thereof (e.g., the information density mapping tool 112, thecognitive load index generation tool 114, the behavioral intentestimation tool 116, the color-emotion analysis tool 118, the emotionalresonance scoring tool 120, the visual attention likelihood estimationtool 122, and the design personalization tool 124) may be implemented onrespective distinct processing platforms, although numerous otherarrangements are possible. For example, in some embodiments at leastportions of the information density toolkit system 110 and one or moreof the client devices 102, the IT infrastructure 105 and/or theinformation database 108 are implemented on the same processingplatform. A given client device (e.g., 102-1) can therefore beimplemented at least in part within at least one processing platformthat implements at least a portion of the information density toolkitsystem 110.

The term “processing platform” as used herein is intended to be broadlyconstrued so as to encompass, by way of illustration and withoutlimitation, multiple sets of processing devices and associated storagesystems that are configured to communicate over one or more networks.For example, distributed implementations of the information processingsystem 100 are possible, in which certain components of the systemreside in one data center in a first geographic location while othercomponents of the system reside in one or more other data centers in oneor more other geographic locations that are potentially remote from thefirst geographic location. Thus, it is possible in some implementationsof the information processing system 100 for the client devices 102, theIT infrastructure 105, the IT assets 106, the information database 108,and the information density toolkit system 110, or portions orcomponents thereof, to reside in different data centers. Numerous otherdistributed implementations are possible. The information densitytoolkit system 110 can also be implemented in a distributed manneracross multiple data centers.

Additional examples of processing platforms utilized to implement theinformation density toolkit system 110 and other components of theinformation processing system 100 in illustrative embodiments will bedescribed in more detail below in conjunction with FIGS. 25 and 26 .

It is to be understood that the particular set of elements shown in FIG.1 is presented by way of illustrative example only, and in otherembodiments additional or alternative elements may be used. Thus,another embodiment may include additional or alternative systems,devices and other network entities, as well as different arrangements ofmodules and other components.

It is not currently feasible to quantitatively map, measure and test howeffectively digital designs transmit information to a viewer, especiallyat scale. Without accurate metrics mapping and measuring informationdensity, customers or other end-users can find themselves either under-or overburdened by the amount of information being provided within aproduct design. The information density toolkit system 110 provides theinformation density mapping tool 112 which implements functionality forautomating the mapping of the density of information a user willexperience when viewing a particular input design (e.g., a productwebpage). The information density mapping tool 112 measures the amountof information within the design and provides quantitative statisticaloutput, which can be used to predict or compare the effectiveness ofmany different product designs prior to launch and to personalize ourdigital experience in a scalable manner. Such output may be used in orby various other tools of the information density toolkit system 110,such as the cognitive load index generation tool 114, the behavioralintent estimation tool 116, the color-emotion analysis tool 118, theemotional resonance scoring tool 120, the visual attention likelihoodestimation tool 122, and the design personalization tool 124.

The Return on Investment (ROI) of the information density mapping tool112 solutions described herein is high, due to the potential foroptimizing direct (e.g., conversion) and indirect (e.g., UCX, CLTV)revenue from product design, especially given the information densitymapping tool 112's ability to facilitate more rapid increases in designiteration throughout the design process. Furthermore, the capacity forthe information density mapping tool 112 of the information densitytoolkit system 110 to be leased to outside parties makes it a potentialsource of direct revenue for an operator of the information densitytoolkit system 110.

Consider, as an example, a “product” or design input that is a checkoutscreen of a webpage or web application. The component features of thisproduct (e.g., a “Hero Banner” of the webpage) are built by design teamswith the ultimate goal of transmitting visual information to viewers(e.g., customers or other end-users). However, human visual perceptionand cognition have natural limitations on the amount of information thatcan be perceived and processed at any given moment. Designers do notcurrently have a means of measuring or predicting how effectively adesign will transmit information to viewers, which prevents designersfrom creating a scalable, personalized digital experience for customersor other end-users.

The amount of information that can be perceived at any given time islargely dependent on the quantity and quality of the content at hand.The semantic content of this information can take many forms, such as:company or product information; procedural information; socialinformation; etc. The amount of information on a screen or within aspecific area of interest (AOI) can vary from sparse to rich, dependingon the design at hand. Likewise, the semantic characterization of theinformation can range from entirely homogenous (e.g., a banner withproduct information only) to extremely heterogenous (e.g., a banner withproduct information, social ratings, procedural information onpurchasing, etc.). The quantity and quality of information on a screenare representative of the cognitive demands placed on the viewer, thusserving as predictors of the following: cognitive load; findability;accessibility; usability; information recall; customer satisfaction;emotional resonance; etc. Effective digital design functions to minimizea viewer's cognitive load and maintain it at reasonable levels, whichmakes the most of his or her limited human cognitive capacity byfacilitating the maximum potential for information retrieval andeffective processing.

Conventional approaches lack the ability to quantitatively map theamount and type of information transmitted to a user during visualinspection. Designers are therefore unable to measure the cognitivedemands placed on viewers. The launch of suboptimal content design hasthe potential to negatively impact financials and user experience.Various tools of the information density toolkit system 110, includingbut not limited to the information density mapping tool 112, can solvethese and other issues by allowing researchers to measure and predictthe cognitive load imparted on viewers pre-launch.

Conventional approaches can also be difficult to scale. Althoughusability and AB testing can be extremely powerful, implementation andanalysis require, on average, two weeks per test. Furthermore, a newtest or test condition must be implemented for each webpage or componentfeature of interest. It is therefore not tenable to test more than fiveor six potential “recipes” prior to launch. As such, it is not feasibleto test many variations in design features and layouts in a systematic,controlled way without great time, effort and resource expenditure. Theinformation density toolkit system 110 enables incorporation of anautomated, quantitative analysis either: as governance into the designsystems language itself; or as a pre-launch analytic toolkit fordesigners. The information density toolkit system 110 can thereforeremedy these and other problems, enabling designers to experiment in ascalable, proactive way.

Conventional approaches further lack the ability to analyze and comparethe cognitive demands that different types of information (e.g., textvs. images, social vs. non-social images, etc.) have on a viewer. Thecognitive load a viewer experiences can vary according to the type andmodality of information (e.g., social vs. technical information). Theinformation density toolkit system 110 can advantageously provide anautomated method for classifying the semantic content of informationwithin a digital design, aiding designers in optimizing viewers'cognitive load while interacting with their designs. Additionally, theaccrual of classified semantic content and associated outcomeascertainment will be a useful resource for designers looking toefficiently experiment. Semantic content classification refers to thecharacterization and grouping of information based on their meaning orother implicit categorical grouping (e.g., faces can be classified asimplicitly “social”).

The information density toolkit system 110 provides the informationdensity mapping tool 112 for automating the detection and quantificationof information contained within a visual stimulus or design (e.g., awebpage or component thereof, a product image, a screen or AOI of anapplication graphical user interface (GUI), etc.). The informationdensity mapping tool 112 can be used to predict and optimize thecognitive demands placed on an individual during visual inspection ofthe visual stimulus. The information density mapping tool 112 thusimparts a scalable means of personalizing digital experience. Textanalysis, area analysis, and semantic categorization of image contentsprovide additional means of information specification and optimization.

FIGS. 2A-2I show a system flow 200 for operation of the informationdensity mapping tool 112. FIG. 2A shows an overview of the system flow200, while FIGS. 2B-2I show examples of the input and output atdifferent steps of the system flow 200. The system flow 200 begins withan input image or design 201, which is illustratively a 2-dimensional(2D) image file (e.g., a *.jpg file, a *.png file, etc.) or pre-rendercode (e.g., Java script). In the description below, the input image 201is assumed to be a color image file. The input image 201 is nextconverted to a greyscale image 203 for improved feature detection andanalysis (e.g., such that other information may be overlayed over thegreyscale image 203 in color to highlight various features thereof).FIG. 2B shows an example of an input image 201 that is converted intogreyscale image 203.

Feature detection 205 is then performed. Feature detection 205 includescalling one or more functions that automatically detect and catalogpoint features within the input image 201 and/or the greyscale image203. Feature detection 205 may include detection of many features, witha subset (e.g., the strongest detected features) being selected and runthrough one or more algorithms with optional specifications. Featuredetection 205 can be performed using one or more algorithms, includingbut not limited to: detection of corner points (e.g., usingHarris-Stephens and/or minimum eigenvalue algorithms, etc.); detectionof key points (e.g., using an Oriented Features from Accelerated SegmentTest (FAST) and Rotated Binary Robust Independent Elementary Features(BRIEF) algorithm); detection of blob features (e.g., using a Speeded-UpRobust Features algorithm); etc. Once feature detection 205 isperformed, the strongest feature points are selected and plotted toproduce detected features 207 as shown in FIG. 2C.

A grid is then overlayed over the input image 201 or greyscale convertedimage 203 to produce overlay grid 209, which is shown in FIG. 2D. Theoverlay grid 209 includes a grid of equally spaced bins. Next, asummation of overlapping features is performed to generate a sum ofoverlaps within the grid squares 211 as shown in FIG. 2E. Summation ofoverlapping features may include transforming point densities intobivariate histogram counts by summing arrays across equally spaced bins.

Subsequently, gridded heatmaps representing 2D and three-dimensional(3D) histograms of information density (e.g., feature densities) withinthe input image 201 are generated and output as the 2D heatmap of counts213 shown in FIG. 2F, and the 3D heatmap of counts 215 shown in FIG. 2G.The 2D heatmap of counts 213 shows densely populated grid squares withlighter shades of grey, and sparsely populated grid squares (e.g., whitespace) with darker shades of grey. The 3D heatmap of counts 215 usespeaks to represent densely populated grid squares and valleys torepresent sparsely populated grid squares.

Overlay processing 217 is then performed to generate a heatmap overlay219, shown in FIG. 2H. The overlay processing 217 produces a smoothedheatmap overlaying the input image 201 by: representing the pointdensities in a mesh grid; interpolating point densities across the meshgrid to smooth corners; setting alpha of null grids to zero to allow theoriginal input image 201 to shine through; and plotting the interpolatedalpha data mesh grid over the input image 201. The heatmap overlay 219provides a visualization tool, highlighting areas of “too little” or“too much” information density to designers and researchers.

Statistical analysis of hotspots 221 is then performed, as shown in FIG.2I. A “hotspot” illustratively refers to a cluster of points within aspatial area of a design. In some embodiments, the statistical analysisincludes density-based spatial cluster analysis. If no area(s) ofinterest or feature(s) of interest are specified by a user, the FIG. 2system flow may end, with the output thereof being used in generatingone or more of: a Cognitive Load Index (CLI) score using the cognitiveload index generation tool 114; behavioral intent estimations using thebehavioral intent estimation tool 116; a map of color-emotion analysisusing the color-emotion analysis tool 118; emotional resonance scoresusing the emotional resonance scoring tool 120; a user-defined AOI mapfor visual attention likelihood using the visual attention likelihoodestimation tool 122; design personalization using the designpersonalization tool 124; etc. Various other descriptive measures andmetrics may be output, including but not limited to: the mean value ofthe entire point density matrix; an information to whitespacesignal-to-noise ratio (e.g., calculated as the count of null matrixelements, divided by the count of nonzero matrix elements); etc. Thestatistical analysis of hotspots 221 may include automaticidentification of clusters (e.g., by color as shown in FIG. 2I). Theoverlay processing 217 may also result in various metrics 223, such asdescriptive density statistics, a number of clusters, the size ofclusters, the distance of clusters, etc.

The information density mapping tool 112 may be further specified, forexample, by an end-user delineating one or more features of interest.The features of interest may include, but are not limited to: one ormore strings of text (e.g., “Black Friday Sale”); one or more areas ofthe image (e.g., upper-left quadrant); one or more semantic categories(e.g., faces); etc. Such input features of interest subsequentlyinitiate a cascade classifier that attempts to further specify therequested informational content within the input image 201 for suchspecified features of interest. If any such features of interest can befound within the input image 201, the output of this specification isone or more areas of interest (AOIs). A graphical display of the AOIswithin the input image 201 is then returned. Various specific measuresmay also be returned as output based on the user specifications, asdetailed below.

An end-user may optionally input one or more text strings in order toidentify one or more AOIs (e.g., rectangular AOIs) matching one or moreof the specified text strings within the input image 201. For each AOIthat is identified, one or more of the following measures may becomputed: the CLI score within the AOIs, and for the entire input image201; descriptive statistics of individual components of informationdensity within the AOIs, and for the entire input image 201; acomparison of CLI scores in the AOI compared to other specified AOIs;and the predicted likelihood of a fixation sequence to fall within theAOIs.

The end-user may also or alternatively optionally input the coordinatesof one or more AOIs within the input image 201. For each AOI that isidentified, one or more of the following measures may be computed: theCLI score within the AOIs, and for the entire input image 201;descriptive statistics of individual components of information densitywithin the AOIs, and for the entire input image 201; a comparison of CLIscores in the AOI compared to other specified AOIs; and the predictedlikelihood of a fixation sequence to fall within the AOIs.

The end-user may further or alternatively optionally input one or moresemantic categories to be identified within the input image 201. One ormore machine learning algorithms may be trained on the user-specifiedsemantic content, such as by using pairs of “Yes” and “No” folders. Foreach pair, the “Yes” folder contains images of a given semantic categoryto be identified (e.g., human faces), while the “No” folder containsimages not associated with the given semantic category (e.g., householdobjects). This type of semantic categorization may also be used tocategorize the semantic content of individual words (e.g., social:“friend”, “we”, etc.) against non-category words (e.g., non-social:“purple”). The information density mapping tool 112 will subsequentlyoutput a graphical display with rectangular boxes surrounding anyidentified AOIs. For each AOI that is identified, one or more of thefollowing measures may be computed: the CLI score within the AOIs, andfor the entire input image 201; descriptive statistics of individualcomponents of information density within the AOIs, and for the entireinput image 201; a comparison of CLI scores in the AOI compared to otherspecified AOIs; the predicted likelihood of a fixation sequence to fallwithin the AOIs; a detailed breakdown of semantic contentclassification; and an emotional resonance score.

The information density mapping tool 112 can advantageously providequantitative analysis of information density in digital design. Theinformation density mapping tool 112 can quantify and map the amount ofinformation conveyed from a digital design. This allows for quantitativeanalysis and simultaneous comparison across multiple digital designs,which in turn makes predictive analyses and assessment of cognitive loadpossible (e.g., for use in generating CLI values using cognitive loadindex generation tool 114, behavioral intent estimations usingbehavioral intent estimation tool 116, color-emotion analysis usingcolor-emotion analysis tool 118, emotional resonance scores usingemotional resonance scoring tool 120, visual attention likelihoodestimations using visual attention likelihood estimation tool 122,design personalization using design personalization tool 124). This canbe used to improve the customer experience (e.g., by reducing cognitiveload, increasing findability, etc.).

The information density mapping tool 112 described herein also providesa fully automated solution, which means that the information densitymapping tool 112 can test and/or compare a vast number of potentialdesigns on the order of seconds or minutes, rather than days or weeks.This represents an impactful increase in the capacity for iterating andimproving on design. It also represents a scalable means ofpersonalizing digital experience. For example, an individual withaccessibility needs could benefit by a personalized digital experiencethat limits the cognitive load of digital content. Thus, output of theinformation density mapping tool 112 can also be used to generate designpersonalization using the design personalization tool 124 (e.g.,automatically rescaling webpage layouts to adjust the density ofinformation on an individual basis).

The information density mapping tool 112 also provides automated methodsof classifying the semantic content within a digital design. In someembodiments, a machine learning-based solution allows designers tofurther optimize cognitive load by adjusting how homogenous orheterogenous the semantic content of a design is (e.g., using the designpersonalization tool 124). Users of the information density mapping tool112 will further benefit by being able to predict emotional resonance,and better assess potential viewers' cognitive load.

Conventional approaches for measuring a product's (e.g., a digitaldesign's) effectiveness include A/B and user testing. Such conventionalapproaches allow designers to test their design “recipes”, use feedbackto iterate on designs, and ultimately place a winning recipe intoproduction. However, A/B tests and user testing are time-consuming anddo not scale well. The information density mapping tool 112 describedherein can advantageously be used, prior to testing, in order to reducethe testing iteration cycle and improve customer or other end-userexperience.

Ideally, product design offers users an experience that avoids bothcognitive overload and cognitive boredom. The information densitymapping tool 112 described herein can map and measure the amount ofinformation on a page or other portion of a digital design in order toincrease one or more of: findability; usability; accessibility; customersatisfaction; engagement; information retention; and emotionalresonance. By improving one or more of these and other factors, theinformation density mapping tool 112 can provide a key competitiveadvantage in positively impacting conversion rates (e.g., as there is asignificant correlation between cognitive load and conversion rates).Furthermore, the information density mapping tool 112 can be offered asa service to various companies or other entities to provide a directrevenue generator for the operator of the information density toolkitsystem 110.

The information density mapping tool 112 described herein also providesa solution that can save a company or other entity indirectly byspeeding up product design iteration, avoiding opportunity costsassociated with launching of suboptimal designs, and avoiding other,more time-consuming methods of design testing. Further, the informationdensity mapping tool 112 described herein represents a cutting-edgemethod of differentiation by using automation at scale. The potentialapplications of the information density mapping tool 112 can also extendbeyond digital design, allowing for adaptively tailoring products andservices to individual customer's unique cognitive needs. Productdifferentiation, personalization, unification, and accessibility effortscould all benefit from the metrics made available by the informationdensity mapping tool 112.

FIGS. 3A and 3B shows examples 300 and 305 of two versions of a “HeroBanner” for a Black Friday promotional webpage. The example 300 has aCLI score of 23.6, while the example 305 has a CLI score of 18.2. Insome embodiments, CLI scores have a numeric output ranging between 1 and100, with the values predicting the cognitive demands a given designwill place on a viewer. Thus, example 305 places less cognitive load onthe viewer than does example 300. Both the examples 300 and 305 have thedensest information at the bolded promotional content “Lowest Prices ofthe Year” and at the eyes of the woman in the banner image. In bothexamples, the Call-to-Action (CTA) buttons for “PC Deals” and“Electronic Deals” lie within hotspots. FIGS. 3A and 3B illustrate howthe information density mapping tool 112 may be used to compare andrefine a digital design.

FIG. 4 shows an example 400 view of an Online Sales Counselor (OSC)configurator tool, which may be used by a sales team of an enterprise.It is clear from the extremely high density of information mapped in theexample 400 that the members of the sales team must deal with highcognitive load (e.g., a CLI score of 86.4) while using the OSCconfigurator tool. This implies that a redesign of the OSC configuratortool interface should incorporate fewer clusters of information, smallerclusters of information, clusters that are well-delineated bywhitespace, and information that is hierarchically organized (e.g., withdifferentiated font sizes and overall size of the information area).

An exemplary process for information density mapping of visual stimuluswill now be described in more detail with reference to the flow diagramof FIG. 5 . It is to be understood that this particular process is onlyan example, and that additional or alternative processes for informationdensity mapping of a visual stimulus may be used in other embodiments.

In this embodiment, the process includes steps 500 through 508. Thesesteps are assumed to be performed by the information density toolkitsystem 110 utilizing the information density mapping tool 112 and thedesign personalization tool 124.

The process begins with step 500, obtaining an input visual stimulus.The input visual stimulus may comprise, for example, an image file. Theimage file may be of a product, an application screen or portionsthereof, a website or webpage or portion thereof, a document, etc., thatis to be analyzed for its associated information density. In step 502,feature points in the input visual stimulus are detected. Step 502 mayutilize at least one of a corner point detection algorithm, a key pointdetection algorithm, and a blob feature detection algorithm.

Densities of the detected feature points in each of the two or moredistinct regions of the input visual stimulus are identified in step504. Relative information density in the two or more distinct regions ofthe input visual stimulus is determined in step 506. Step 506 maycomprise calculating at least one of: a mean value of a point densitymatrix of the detected feature points; and an information to whitespacesignal-to-noise ratio, the information to whitespace signal-to-noiseratio being calculated as a count of null elements in the point densitymatrix divided by a count of nonzero elements in the point densitymatrix. Step 506 may also or alternatively comprise performing adensity-based spatial cluster analysis of the detected feature points toidentify statistical hotspots representing ones of the two or moredistinct regions of the input visual stimulus having an informationdensity exceeding a designated threshold.

In some embodiments, step 504 comprises overlaying a grid of equallyspaced bins over the input visual stimulus and generating bivariatehistogram counts for the bins of the overlay grid. The bivariatehistogram counts represent numbers of the detected feature points ineach of the bins of the overlay grid. Step 506 may comprise generatingat least one information density heatmap representing the generatedbivariate histogram counts for the bins of the overlay grid. The atleast one information density heatmap may comprise a two-dimensionalheatmap of the grid of equally spaced bins. The at least one informationdensity heatmap may also or alternatively comprise a three-dimensionalheatmap of the grid of equally spaced bins in which peaks of thethree-dimensional heatmap represent densely populated ones of the binsand valleys represent sparsely populated ones of the bins. The at leastone information density heatmap may be generated by representing theidentified densities of the detected feature points in a mesh gridsurface, interpolating the identified densities of the detected featurepoints across the mesh grid surface to generate an interpolated meshgrid surface that smooths corners, setting an alpha value of null gridsin the interpolated mesh grid surface to zero to generate aninterpolated alpha mesh grid surface, and plotting the interpolatedalpha mesh grid surface over the input visual stimulus.

In some embodiments, step 506 comprises determining, for one or moreuser-specified features of interest, at least one of: cognitive loadindex scores for one or more areas of interest of the input visualstimulus containing at least one of the one or more user-specifiedfeatures of interest; descriptive statistics of individual components ofinformation density within the one or more areas of interest; acomparison of the cognitive load index scores across different ones ofthe one or more areas of interest; and a predicted likelihood of afixation sequence to fall within the one or more areas of interest. Theuser-specified features of interest may comprise at least one of: one ormore text strings of interest; coordinates for at least one of the oneor more areas of interest; and one or more semantic categories ofinterest. Step 506 may further comprise generating a visualization ofthe input visual stimulus highlighting the one or more areas ofinterest.

The FIG. 5 process continues with step 508, modifying a design of theinput visual stimulus to adjust the relative information density amongat least two of the two or more distinct regions of the input visualstimulus. Step 508 may comprise at least one of: modifying content in atleast a given one of two or more distinct regions of the input visualstimulus; moving content from a first one of the two or more distinctregions of the input visual stimulus to a second one of the two or moredistinct regions of the input visual stimulus; and adjusting a distancebetween the at least two of the two or more distinct regions of theinput visual stimulus.

Currently, designers and researchers do not have a method to measure thecognitive demands imparted on an individual while viewing andinteracting with digital content. Compared to a webpage with sparselypopulated content, webpage content that is very densely populatedrequires a greater amount of cognitive energy in order for a viewer toprocess the information and complete an intended task. Too manypotential fixation points can lead to attentional blindness at one ormore features where attention is intended (e.g., viewers fail to findand interact with an intended CTA). Conversely, too little contentwithin a page can lead to cognitive boredom, increasing page exitsand/or the number of pages needed for a user to complete a task. Thetechniques described herein provide a novel quantitative measure forcomputing cognitive load, referred to as the Cognitive Load Index orCLI, generated using the cognitive load index generation tool 114. CLIvalues may be used in various other tools of the information densitytoolkit system 110. For example, the CLI values may be used as an inputfor the visual attention likelihood estimator tool 122 and the designpersonalization tool 124. The CLI values may also be part of an outputof the information density mapping tool 112, etc. The CLI values may beused for optimizing digital design to enhance information transmission,retention, and usability for end-users. Given that each individual isunique in his or her cognitive capacity, CLI values allow for scalable,neurodiversity-based personalization for digital experience.

The novel solutions for generating CLI values described herein representan automated method of statistically analyzing digital content todetermine its potential for placing cognitive demands on our users. Asnoted above, CLI values may be provided as a quantitative output of theinformation density mapping tool 112, which maps the density ofinformation contained within a digital design. The CLI is illustrativelya numerical output that can range from 1 to 100, representing the rangefrom extremely sparse content to extremely dense content within adesign. The goal of the CLI is to anchor the optimization of digitalcontent, transmission, and retention of information in a standard,interpretable scale. In turn, designers who use the CLI will be betterable to facilitate task completion (e.g., reduce negative sentiment,errors and exits) and to enhance digital experiences (e.g., increaseusability metrics, conversion rates, and long-term customer value) forindividual end-users.

The visual complexity of a digital design directly affects how a usercan cognitively process and interact with the content. Visuallyprocessing a large amount of densely packed digital content represents adifficult task for a user's cognitive system. Conversely, there is arisk of cognitively boring users when a design's content is too sparse.Without a quantitative method for measuring the potential demands adigital design might place on users' cognitive processes, designers areunable to predict and optimize digital content to better meet users'individual needs.

Conventional cognitive demand assessments require in-person laboratoryassessment of physiologic responses to visual stimuli. Conventionalremote user research techniques lack the ability to quantify the amountof information transmitted to a user during visual inspection. Suchconventional approaches are therefore unable to measure the cognitivedemands placed on users. The launch of suboptimal content design has thepotential to negatively impact financials and user experience.Development of the cognitive load index would solve this issue byallowing researchers to measure and predict the cognitive load impartedon viewers.

There is currently no automated way to predict a user's cognitive loadresponse to digital content. Some conventional approaches for measuringcognitive load rely on eye-tracking and other in-person biometricmeasures. These approaches, however, cannot be automated or utilizedoutside of research laboratories and therefore do not scale well.Conventional approaches also have no means of optimizing digital contentto an individual user's cognitive abilities. Without knowledge of how anindividual's brain might be able to process visual information (e.g., anindividual with accessibility requirements might perform better withsparser content), designers cannot personalize digital experience tousers' specific cognitive needs.

In some embodiments, the cognitive load index generation tool 114 takesas input a matrix of values outputted from the information densitymapping tool 112. The input matrix represents the density of informationwithin the digital design inputted into the information density mappingtool 112. In order to facilitate simple comparability and interpretationof cognitive load, several statistical measures based on the spatialclustering of the input matrix may be combined into a single indexvalue. Spatial autocorrelation statistics will be used to assess theclusters' size, number, and distance to evaluate whether a given patternof content is clustered, dispersed, or random. As spatialautocorrelation is an inferential statistic, it is interpreted in thecontext of the null hypothesis (H₀). FIG. 6 shows a table 600, providinga guide for interpretation of possible combinations of results. Ineffect, the higher or lower the z-score, the more intense theclustering. A z-score near 0 indicates no apparent spatial clustering.The cognitive load index or CLI may be represented as a single numericalvalue, calculated as described below.

Cluster analysis is performed based on an information density mappingmatrix. The cluster analysis may include identifying a number ofclusters, the distance between clusters, and the size of clusters. Insome embodiments, a normalized number of clusters, N, is generated(e.g., using a minimum-maximum scalar method) by performing a clusteringalgorithm (e.g., a density-based clustering algorithm) on theinformation density mapping matrix. The number of clusters identified bythe algorithm are then normalized such that:

$X_{norm} = \frac{X - X_{\min}}{X_{\max} - X_{\min}}$

The normalized cluster number, {circumflex over (N)} will, therefore,fall within the range of [0,1]: x₁={circumflex over (N)}. In someembodiments, a hypothesis is used that the larger the number ofclusters, the higher the cognitive load will be on an individual.

To determine the distance between clusters, a distance metric may becalculated between the generated clusters. In some embodiments, theaverage normalized Euclidean distance, D, between all possible clustercentroid pairs is calculated by:

-   -   1. Identifying the centroid of each cluster;    -   2. Calculating the Euclidean distance between all possible        centroid pairs;    -   3. Normalizing the Euclidean distances such that:

${X_{norm} = \frac{X - X_{\min}}{X_{\max} - X_{\min}}},$

where the normalized Euclidean distances will, therefore, fall withinthe range of [0,1]; and

-   -   4. Taking 1 minus the average of all normalized Euclidean        distances, {circumflex over (D)}:

x ₂=1−{circumflex over (D)}

In some embodiments, a hypothesis is used that the smaller the averagedistance between the cores of each cluster, the higher the cognitiveload will be on an individual.

To determine the size of the clusters, some embodiments calculate 1minus the average normalized cluster size, Ŝ, by:

-   -   1. Identifying the centroid of each cluster;    -   2. Drawing the lines to form a polygon around the border cluster        points;    -   3. Calculating the area within each polygon;    -   4. Normalizing the cluster areas such that

${X_{norm} = \frac{X - X_{\min}}{X_{\max} - X_{\min}}};$

and

-   -   5. Taking 1 minus the average of all normalized sizes of the        clusters, Ŝ:

x ₃=1−Ŝ

In some embodiments, a hypothesis is used that the larger the averagesize of the cluster, the higher the cognitive load will be on anindividual.

Cognitive load may also be based on optional text complexity scoreinputs. End-users may have the option to manually input a textcomplexity/readability score. This input is not required, but mayimprove the accuracy of the cognitive load index score. The user shouldinput the readability score associated with the original input image, aswell as the possible range that the score can take (e.g., 1-100). Theaverage normalized readability score, Ĝ, may be calculated by:

-   -   1. Normalizing the inputted score based on the inputted maximum        and minimum range between 0 and 1, such that:

${X_{norm} = \frac{X - X_{\min}}{X_{\max} - X_{\min}}};$

and

-   -   2. If no user input is provided, Ĝ=0 and w_(G)=0:

x ₄ =Ĝ

In some embodiments, a hypothesis is used that the higher the textcomplexity, the higher the cognitive load will be on an individual.

Each of the elements {circumflex over (N)}, {circumflex over (D)}, Ŝ,and Ĝ may be weighted, with the weights being predefined or optionallymanually inputted by a designer or other end-user. If no weights areinputted or defined, the weights may default such that each element willbe assigned an equal weight. The cognitive load index or CLI value canthen be calculated as follows:

${CLI} = {\frac{1}{n}{\sum\limits_{i = 1}^{n}{w_{i}x_{i}}}}$

where n=4 if a readability score, Ĝ, is available, and n=3 if Ĝ is notavailable. The higher the CLI value, the higher the cognitive burdenplaced on an individual viewing the content. As such, relatively highand relatively low values represent less desirable digital designscompared with those that receive a midpoint value.

The cognitive load index generation tool 114 may output a plot of theranges of visual complexity for a given design. The plot shows CLIscores against individual webpage interaction metrics, with the plotbeing utilizable for helping guide designers and researchers toward anoptimal mix of interest, content and information density. FIG. 7 showsan example plot 700, which provides a range of validated “targets” fordesigners and should accompany a library of sorts in which simple,neutral and complex design examples are cataloged for study andreference. Learnings from different combinations of cluster size,distance and count with associated CLI scores will allow for purposefulexperimentation. Designers can reference and learn the types andcombinations of pictures and text that lend themselves to better CLIscores along with better engagement and performance in a real-worldsetting (e.g., online e-commerce).

In order to validate the accuracy, precision, and interpretation of theCLI and its component measures, various methods can be employedincluding but not limited to: mouse-tracking and engagement metrics;platform analytics metrics; remote eye-tracking; and physiologicvalidation. Third-party consumer analytic tools (e.g., Adobe Omniture,ContentSquare, etc.) may be used to create a heat map that shows realuser hovers, clicks, and individual mouse-based engagement with thedigital input. The hovers and clicks from the heatmap are indicative ofactual eye motion, so this heatmap will act as a proxy for aphysiological assessment of eye-tracking motion. The interactive outputwill be compared against the information density heatmap to determinethe overlap between the hypothesized areas of engagement with the actualareas of engagement. Acceptance or rejection of initial hypothesesaround predicted individual engagement with the content can then bemade.

Further engagement and individual content interaction metrics may betaken from platform analytics tools (e.g., an Adobe Cloud Analyticssuite) to measure variables such as page clicks, pathing, time spent onpage, overall engagement, conversion rates, etc. These metrics will becompared against the CLI and its component measures to assess how andthe degree to which the number, spacing, and size of the algorithmicallyproduced clusters affect the individual's engagement with content. Thisshould act as a proxy for the cognitive load on the individualsinteracting with the content and aid in the interpretation of simple,neutral, and more complex designs. More specifically, the analyticsplatform metrics will be used to evaluate the digital designs'effectiveness and are expected to correlate with the CLI.

Using web-based eye-tracking software, researchers can remotely captureusers' eye-movements while remotely (e.g., not in an in-personlaboratory) viewing the digital design input. The eye-trackingcoordinates and blink rates can then be compared to the informationdensity heatmap and CLI for validation. The eye-tracking coordinateswill be compared against the information density heatmap to determinethe overlap between the hypothesized areas of visual attention with theactual areas of visual attention. The blink rates will be compared withthe CLI, whereby a positive linear relationship is hypothesized (e.g.,higher blink rates are associated with higher cognitive load).

The gold standard for confirmation or rejection of the hypotheses usedin some embodiments would be done with in-person laboratory-grademeasurement of individuals' physiologic responses to their interactionwith the visual content. These measurements could include eye-tracking,pupil dilation, blink rate, and temperature assessments throughouttask-oriented interactions with the inputted digital content anddesigns.

Conventional approaches for measuring cognitive load rely on in-personlaboratory-based research methods, such as eye-tracking glasses andphysiological measures. Researchers often use blink rate as a metric ofcognitive load. The solutions described herein differ from conventionalapproaches in that the solutions described herein are predictive andautomated. The CLI solutions described herein use inputs from a visualstimulus to predict cognitive load prior to testing, rather thanmeasuring it post hoc in a research setting. Furthermore, the CLIsolutions described herein are fully automated, which means they canscale to analyze any product or other design image in seconds,facilitating rapid iteration in testing and design.

The cognitive load index generation tool 114 provides a quantifiableanalysis of a digital design's potential for exerting cognitive demandson a user. The quantitative assessment enables designers and researchersto measure and compare the potential effectiveness of digital designs.This can be used to improve the customer experience by reducingcognitive load or cognitive boredom. The cognitive load index generationtool 114 further provides scalable and automated prediction of thecognitive load that a user will experience while interacting withdigital designs and content. The cognitive load index generation tool114 provides an algorithmic approach which, once validated, does notrely on in-laboratory testing or user input. It is therefore easilyscalable and can be used to automatically assess and compare thepotential effectiveness of any digital design. The cognitive load indexgeneration tool 114 described herein further enablesneurodiversity-based personalization (e.g., using the designpersonalization tool 124). Every individual has a unique capacity forcognitive processing. The cognitive load index generation tool 114allows for digital design to be personalized to each individual's uniqueneeds and preferences.

The cognitive load index generation tool 114 represents an opportunityfor differentiation from conventional approaches by tailoring better,smarter experiences and designs. There is competitive advantage inutilizing the scalable methods for generating CLI metrics provided usingthe cognitive load index generation tool 114 for predicting andfacilitating customer success. These techniques may be used in amultitude of industries and use cases, such as companies or entitiesthat endeavor to create product designs that reflect “cognitiveergonomics” (e.g. similar to endeavors for creating ergonomic workplacesfor employees). Making products and decision environments simpler andeasier for customers or other end-users, and even personalized to eachindividual's unique cognitive capacity, is a customer-centric futurethat various entities across many industries strive to attain.

In general, there is established value in assessing behavioral metrics.Behavioral metrics are objective and can be collected implicitly whilethe user is completing their task and without overt collectionactivities (e.g., stopping to ask the user to provide a subjectiverating of ease of use). Individuals tend to have subconscious biaseswhich prevent them from self-reporting with any fidelity, whereasbehavioral metrics provide a more objective measure of customersentiment. There is evidence that shows that these kinds of behavioralfeatures can reflect mental states, such as mental effort and cognitiveload. There is considerable margin for differences within and betweenusers. Understanding behavioral markers of individual variance incognitive capacity allows for tailoring tools provided by theinformation density toolkit system 110, including the cognitive loadindex generation tool 114, to an individual's needs (e.g., accessibilityneeds), which is likely to enhance CLTV and loyalty. These metrics arealso distinct from physiological measures in that they are mostly orentirely under the users' voluntary control and may, therefore, besubject to intervention by website design. In sum, a behavioral metricfor cognitive load prediction will provide researchers and designers amore accurate and reliable measure (e.g., compared to self-reporting),which can be used to make personalized, human-centric improvements todigital design. This is hypothesized to lead to increases in direct(e.g., conversion rates due to enhanced in-page performance) andindirect (e.g., CLTV, loyalty) sources of revenue.

The correlation between CLI scores and conversion rates for designs(e.g., webpages) is investigated by inputting various product webpagesinto the cognitive load index generation tool 114 to calculate the CLIper page. CLI scores ranged between 52.3 and 76. Two examples areillustrated in FIGS. 8A-8B and 9A-9B. FIG. 8A shows an input webpage 800(e.g., a product webpage for Dell Precision desktops) along with aheatmap overlay thereof 805 with a CLI of 55. This suggests the overallcognitive load of the input webpage 800 is moderate. FIG. 8B shows aplot 810 of the output of automated cluster analysis for the inputwebpage 800, with cluster centers marked by asterisks. FIG. 9A shows aninput webpage 900 (e.g., a product webpage for Alienware laptops) alongwith a heatmap overlay thereof 905 with a CLI of 67.6. This suggests theoverall cognitive load of the input webpage 900 is tending towards high.FIG. 9B shows a plot 910 of the output of automated cluster analysis forthe input webpage 900, with cluster centers marked by asterisks. FIG. 10shows a plot 1000, illustrating the results of a generalized linearmodel indicating that CLI scores are a significant predictor (e.g.,p<0.01) on conversion rates across 13 product pages. This model suggeststhat conversion increases with CLI score.

An exemplary process for cognitive load scoring of a visual stimuluswill now be described in more detail with reference to the flow diagramof FIG. 11 . It is to be understood that this particular process is onlyan example, and that additional or alternative processes for cognitiveload scoring of a visual stimulus may be used in other embodiments.

In this embodiment, the process includes steps 1100 through 1106. Thesesteps are assumed to be performed by the information density toolkitsystem 110 utilizing the cognitive load index generation tool 114 andthe design personalization tool 124.

The process begins with step 1100, obtaining an information densitymatrix for an input visual stimulus, the information density matrixcharacterizing information density of feature points in the input visualstimulus. The input visual stimulus may comprise an image file. Theimage file may be of a product, an application screen or portionsthereof, a website or webpage or portion thereof, a document, etc., thatis to be analyzed for its associated cognitive load index.

In step 1102, one or more clusters of feature points in the input visualstimulus are identified by performing spatial clustering of the featurepoints utilizing the information density matrix. A cognitive load scorefor the input visual stimulus is determined in step 1104 based at leastin part on the identified one or more clusters of feature points. Thecognitive load score characterizes cognitive energy required to mentallyprocess the input visual stimulus. In some embodiments, step 1104comprises determining a weighted average of two or more cognitive loadscore components. The two or more cognitive load score components maycomprise two or more of: a normalized number of the identified one ormore clusters; an average normalized distance between the identified oneor more clusters; an average normalized size of the identified one ormore clusters; and an average normalized readability score of the inputvisual stimulus.

Step 1104 may be based at least in part on a number of the identifiedone or more clusters of feature points. The cognitive load score for theinput visual stimulus increases with the number of the identified one ormore clusters of feature points. Step 1102 may comprise performing adensity-based clustering algorithm on the information density matrix togenerate a normalized number of the identified one or more clusters offeature points. The cognitive load score for the input visual stimulusdetermined in step 1104 may be based at least in part on the normalizednumber of the identified one or more clusters of feature points. Thenormalized number of the identified one or more clusters of featurepoints may be generated utilizing a minimum-maximum scalar algorithm.

Step 1104 may comprise calculating distances between the identified oneor more clusters. The cognitive load score for the input visual stimulusincreases as distances between the identified one or more clustersincreases. Calculating the distances between the identified one or moreclusters may comprise calculating Euclidean distances between centroidsof the identified one or more clusters, normalizing the calculatedEuclidean distances, and determining an average of the normalizedEuclidean distances. Step 1104 may also or alternatively comprisecalculating sizes of the identified one or more clusters. The cognitiveload score for the input visual stimulus increases as the sizes of theidentified one or more clusters increases. Calculating the sizes of theidentified one or more clusters may comprise, for each of the identifiedone or more clusters, identifying a centroid of that cluster, drawinglines to form a polygon around border feature points of that cluster,and calculating an area within the polygon, and normalizing the areas ofthe polygons for the identified one or more clusters. Step 1104 mayfurther or alternatively comprise determining text complexity scores forone or more areas of the input visual stimulus.

The FIG. 11 process continues with step 1106, modifying a design of theinput visual stimulus to adjust the cognitive load score of the inputvisual stimulus. Step 1106 may comprise modifying the identified one ormore clusters of feature points to reach a target cognitive load score,the target cognitive load score being personalized for a given end-userviewing the input visual stimulus. Step 1106 may comprise at least oneof modifying a number of the identified one or more clusters, modifyinga distance between a first one of the identified one or more clustersand a second one of the identified one or more clusters, and modifying asize of at least one of the identified one or more clusters.

When designing a product, understanding a user's intent whileinteracting with that product is an important but often difficult task.Knowing a user's intent allows a designer to tailor that user's specificjourney, thereby facilitating a smooth and positive experience. However,a user's intent—and likewise his or her journey—is often anything butsingular and linear. Therefore, the information density toolkit system110 provides the behavioral intent estimation tool 116 as a solution fordynamically updating the predicted likelihood that a user has at anygiven point in time when viewing an inputted visual stimulus. Usingbehavioral inputs and historical user data, the behavioral intentestimation tool 116 can constrain the list of possible user intents,allowing researchers and designers to better focus their product designtoward a behavioral intent-oriented experience.

Establishing an individual's intent at any given time is a particularlydifficult task. As demonstrated in a game of chess, intent is only knownprivately within an individual's own mind in the first instance, but canlater be revealed through that individual's subsequent behavior. Carefulobservation of an individual's actions allows one to establish anestimate that those behaviors will lead to a particular outcome, therebyrevealing the intent behind those actions.

To complicate matters further, a user's intent can change based on thepotential pathways presented to him or her in that moment. To continuethe chess analogy, a player will dynamically change their originalstrategy based on their opponent's actions. Similarly, a user may changehis or her original intent based on the possible options or informationpresented by the current visual stimulus. For example, a user of a newlaptop may originally intend to use that product for browsing theInternet at that time, but may pivot away from that original intentafter seeing an icon for a new software of interest. Another examplemight be a digital user intending to browse for a new monitor, who thengets distracted by a marketing banner indicating new laptops on sale,and subsequently starts searching for specs and reviews for a particularlaptop.

By establishing a pattern of actions that most often leads to aparticular outcome, over many different patterns and outcomes, it ispossible to create an algorithm that sequentially updates the likelihoodof a user's intent at any point in the journey. This also allows thebehavioral intent estimation tool 116 to dynamically observe and predictchanges in intent.

Knowing a user's intent at any given moment allows a product designer tooptimally shape the user's experience. Yet, defining a user's intent atany given moment is a problem of considerable complexity. Conventionalapproaches are not able to achieve a point-in-time estimate of possibleuser intents from an input digital design (e.g., a product image).Therefore, design of the user experience cannot be truly optimized. Theinformation density toolkit system 110 provides a number of tools (e.g.,the information density mapping tool 112, the cognitive load indexgeneration tool 114, the color-emotion analysis tool 118, the emotionalresonance scoring tool 120, the visual attention likelihood estimationtool 122, the design personalization tool 124) which can be used foroptimizing customer or other end-user experience through improved designusing the outputs of the behavioral intent estimation tool 116. Thebehavioral intent estimation tool 116 can enhance various other tools ofthe information density toolkit system 110 by providing an estimation ofuser intent while interacting with designs. The behavioral intentestimation tool 116 can estimate user intent based on user inputs intothe information density toolkit system 110 to fully optimize output.

Various entities endeavor to develop intent engines that algorithmicallycapture individual users' intents, but such intent engines areinsufficient for various purposes. Namely, some embodiments require anintent engine that can calculate behavioral intent based on apoint-in-time digital design (e.g., a product image) and/or pathwayanalysis, rather than based on user cookies or online behavior. Forexample, a product designer may wish to know how changing the number ofbuttons/options on a page will affect user intent. In conventionalapproaches, a designer would need to design and test AB alternatives toarrive at a solution. Such conventional approaches are inadequate atscale, which prevents designers from serially testing and comparing alarge volume of designs. Without behavioral intent estimation, anoptimized output cannot necessarily be provided.

In some embodiments, Bayes' Theorem is used to estimate and subsequentlyupdate the likelihood of a user's intent based on historical userbehavior and possible future options. Given the point in timerepresented by the hypothetical moment that a viewer is first presentedwith a visual stimulus, the program arrives at an estimation byutilizing historical user data as priors and updating with anyidentified object-based pathways within the visual stimulus. Anobject-based pathway is a visual object that represents a pathway, whichmight lead a viewer closer towards fulfilling their original intent. Forexample, a CTA button is a digital object that, when clicked, redirectsthe viewer to a page that may bring them closer to his or her end goal.As another example, a block of text is an object, which after being readby the viewer, fulfills the viewers intent of searching for theinformation contained within that block of text. As a further example, aside view of a laptop in an image can provide the viewer with a visualestimation of the laptop's lightweight design and portability, whichsatisfies the viewer's search for such information. A forward movementpathway is an object-based pathway (e.g., a hyperlink) that allows auser to move forward in their journey. A backward movement pathway(e.g., the back button) is an object-based pathway that allows a user tomove backward one or more steps in their journey.

FIG. 12 shows an example of behavioral intent estimation, illustratingpossible intents as an end-user traverses through a set of productwebpages 1201, 1202, 1203 and 1204. As illustrated, the possiblebehavioral intents narrow as the end-user selects particular options andmoves among the product webpages 1201, 1202, 1203 and 1204. Thebehavioral intent estimation begins by defining a very large set ofpossible behavioral intents. At the beginning of the user's journey atproduct webpage 1201 (e.g., a website's homepage), all potentialbehavioral intents within a previously defined set are possible. It istherefore unlikely for the user's initial visit to the product webpage1201 to have much predictive power, aside from historical behavioralpatterns (e.g., 40% of users typically go from the product webpage 1201to one or more category pages). However, at a certain point in thecustomer journey, it becomes very likely that potential decisionpathways (e.g., CTAs, links, the back button, etc.) accurately constrainthe intent space. In other words, a user's intent can likely be guessedbased on what pathways are available at any given point.

Available pathways are represented by backward and forward movement. Inthe digital space, the back button typically represents either a misstep(e.g., “the link did not provide what I was looking for”), a mid-step(e.g., “I found some, but not all, of the information I was lookingfor”), or an end-step (e.g., “I found the information I was looking for,and now I want to go back and do something else”). Thus, it isadvantageous to include the forward pathways from the prior page in thecurrent page's intent space, albeit weighted in a manner that reflectsthe decreased possibility of them being the true intent. These backwardmovement possibilities, along with those represented by each forwardmovement pathway (e.g., links), can be represented in a weighted fashionwithin the model's intent space. Weights can be assigned based onhistorical user behavior and strength of the pathway stimulus (e.g.,large banner vs. very small CTA). All other intents not represented bybackward and forward movement can be safely rejected as likely intents.

In this manner, it is possible to constrain the intent space and assignweights to each remaining potential intent, arriving at a likelihoodestimate that any given intent is the user's true intent at that momentin time. When only a single time point in this journey is known (e.g.,corresponding to visiting the webpage 1203 in the FIG. 12 example), itis typically not possible to include weighted estimates of backwardmovement intents (e.g., the greyed-out intents in FIG. 12 ), as theprior page is unknown. In this case, the user has the option to manuallyinput the backward movement intents, or the estimation can be calculatedbased solely on forward movement intents.

FIG. 13 shows a system flow 1300 for the behavioral intent estimationtool 116. The system flow 1300 starts with an input image 1301 or othervisual stimulus file (e.g., a *.jpg file, a *.png file, etc.). Thebehavioral intent estimation tool 116 also takes as input a statisticalanalysis output 1303 from the information density mapping tool 112. Thebehavioral intent estimation tool 116 performs text-based analysis 1305,such as by detecting text within the input image 1301. Identifiedcharacters and words are accompanied with confidence levels and stimulusboundary locations for subsequent analyses. Static image analysis 1307is then performed to detect static images within the input image 1301,with the image boundaries being stored for subsequent analyses.

Path detection 309 is then performed for object-path categorization1311, which may include optional specifications, selection of one ormore algorithms, recognizing objects, and backward-forwardsubcategorization. The behavioral intent estimation tool 116 uses userinput, JavaScript input, and/or machine learning semantic classificationto categorize any objects identified during the text-based analysis 1305and static image analysis 1307 as potential decision pathways.Identified object-based pathways are then further subcategorized asforward-movement pathways (e.g., a button that turns a product on, or adigital link that sends a user to a new page) or backward-movementpathways (e.g., the “back” button on a webpage). Object-based pathwayscan be specified by the user, identified from JavaScript (e.g., if it isa web design), or identified algorithmically using one or more machinelearning classifiers. If machine learning is used, libraries of exampleobject-based pathway stimuli (e.g., text or images) for the specificproduct-type should first be accumulated or provided by the user. Adefault list of intents is called by the behavioral intent estimationtool 116, and all intents are initially equally likely. The object-basedpathways specified in this step act as priors, P(A), for the Bayesianupdater. Essentially, the priors modify the default list of intents tospecify whether they are a likely possibility (e.g., intent isrepresented by at least one object-based pathway) or an unlikelypossibility (e.g., no object-based pathways represent the intent).

The list of intents can be further modified based on historical userdata 1313. When available, the user uploads historical user data 1313(e.g., such as from a website homepage, 10% of users use the search bar,8% of users visit a “Deals” page from the “Hero Banner”, etc.). Thehistorical user data 1313 acts as the likelihood function, P(B|A),within the Bayesian updater calculation. Additional optional user input1315 may also be provided. The optional user input 1315 may comprise,for example, user-specified intent outright. The benefit of suchoptional user input 1315 would be to examine how different personas oruser types are represented in the rest of the behavioral intentestimation tool 116 or across different inputs.

Iterative analytics 1317 are then performed based on the object-pathcategorization 1311, the historical user data 1313 and the optional userinput 1315. The iterative analytics 1317 may include Bayesian updating1319 which utilizes the Bayes Theorem:

${P\left( A \middle| B \right)} = \frac{{P\left( B \middle| A \right)} \times {P(A)}}{P(B)}$

to define the conditional probability of a given behavioral intent, A,occurring given that the user is viewing the current stimulus, B. Usingthe probability of a user viewing the stimulus given their behavioralintent, P(B|A) as defined by the historical user data 1313, and theprobability of the behavioral intent as defined by the object-pathcategorization 1311, the behavioral intent estimation tool 116 caniteratively calculate the probability of each behavioral intentoccurring. The final output is a behavioral intent estimation 1321,which is a list of possible intents and their associated probabilities.

Conventional approaches to measuring intent include approaches thatutilize artificial intelligence, machine learning and neural networks toidentify intent based on digital engagement. Such conventionalapproaches can be powerful, but are costly to develop and maintain. Thebehavioral intent estimation tool 116 provides an approach that is lesscostly but is unique in its ability to address a specific need for apoint-in-time estimate of behavioral intent based on an inputted visualstimulus.

The behavioral intent estimation tool 116 described herein provides anovel approach for estimating a user's intent based on an input visualstimulus or design (e.g., a product image, a webpage, etc.) andobject-based pathway analysis. In some embodiments, Bayesian analysis isperformed using identified object-based pathways to update priors andprovide a point-in-time estimate of the user's most likely intent. Thisautomated methodology can be used at scale to provide analysis onhundreds of different design recipes in minutes. This is advantageous toany design or research group interested in quickly testing and comparingproduct designs.

The behavioral intent estimation tool 116 may also advantageously bespecifically designed for the inputs and constraints of the informationdensity toolkit system 110. The behavioral intent estimation tool 116can advantageously take the same input, with the option to prompt theuser for additional input when desirable, and return a user intentestimation that both stands alone and acts as important input to othertools within the information density toolkit system 110, including butnot limited to the visual attention likelihood estimation tool 122. Asdescribed above, the behavioral intent estimation tool 116 can provide akey input that allows designers and researchers across a variety ofindustries to improve how they design products and display information.“Cognitive ergonomics” is a customer-centric future in which variousentities facilitate better information transmission, better products,better decisions, and better experiences.

An exemplary process for behavioral intent estimation for a visualstimulus will now be described in more detail with reference to the flowdiagram of FIG. 14 . It is to be understood that this particular processis only an example, and that additional or alternative processes forbehavioral intent estimation for a visual stimulus may be used in otherembodiments.

In this embodiment, the process includes steps 1400 through 1408. Thesesteps are assumed to be performed by the information density toolkitsystem 110 utilizing the behavioral intent estimation tool 116 and thedesign personalization tool 124.

The process begins with step 1400, obtaining an information densitymatrix for an input visual stimulus, the information density matrixcharacterizing information density of feature points in the input visualstimulus. The input visual stimulus may comprise an image file. Theimage file may be of a product, an application screen or portionsthereof, a website or webpage or portion thereof, a document, etc., thatis to be analyzed for its associated behavioral intent.

In step 1402, content of the input visual stimulus is analyzed utilizingthe information density matrix to identify one or more objects in theinput visual stimulus. Step 1402 may comprise detecting text within theinput visual stimulus and associating at least a given portion of thedetected text with at least a given one of the one or more objects inthe input visual stimulus, the given portion of the detected text havinga stimulus boundary location within the given object in the input visualstimulus. Step 1402 may also or alternatively comprise detecting one ormore images within the input visual stimulus and associating at least agiven one of the one or more images with at least a given one of the oneor more objects in the input visual stimulus, the given image having astimulus boundary location within the given object in the input visualstimulus.

One or more object-based pathways in the input visual stimulus aredetermined in step 1404. Each of the one or more object-based pathwaysis associated with one of the one or more objects in the input visualstimulus. The one or more object-based pathways represent potentialdecision pathways for a user viewing the input visual stimulus to reacha desired result. The input visual stimulus may comprise one of asequence of visual stimuli. At least a given one of the one or moreobject-based pathways may be associated with a given one of the one ormore objects representing a forward-movement pathway or a backwardmovement pathway. The forward-movement pathway, when selected, initiatesdisplay of a subsequent visual stimulus in the sequence of visualstimuli. The backward-movement pathway, when selected, initiates displayof a previous visual stimulus in the sequence of visual stimuli.

In step 1406, probabilities for two or more different behavioral intentsof the user viewing the input visual stimulus are estimated based atleast in part on the one or more object-based pathways. Step 1406 maycomprise calculating conditional probabilities for the two or moredifferent behavioral intents utilizing a machine learning algorithm. Themachine learning algorithm may comprise a Bayes algorithm, and the oneor more object-based pathways modify a default list of possiblebehavioral intents used in the Bayes algorithm. The Bayes algorithm mayutilize a likelihood function that takes as input historical user dataspecifying historical probabilities for the two or more differentbehavioral intents. In some embodiments, the two or more differentbehavioral intents are part of a plurality of possible behavioralintents, with each of the plurality of possible behavioral intents beinginitialized with an equal conditional probability value, and wherein theinitialized conditional probability values of respective ones of theplurality of possible behavioral intents are adjusted based at least inpart on whether at least one of the one or more object-based pathwaysrepresents that possible behavioral intent. Step 1406 may compriseassigning weight values to the two or more different behavioral intentsbased at least in part on at least one of information density of theobjects associated with the one or more object-based pathways and sizesof the objects associated with the one or more object-based pathways.

The FIG. 14 process continues with step 1408, modifying a design of theinput visual stimulus to adjust the estimated probabilities of the twoor more different behavioral intents of the user viewing the inputvisual stimulus. Step 1408 may comprise modifying at least one of theone or more objects in the input visual stimulus to increase anestimated probability of a given one of the two or more differentbehavioral intents relative to other ones of the two or more differentbehavioral intents. Step 1408 may also or alternatively comprise atleast one of modifying information content of at least one of the one ormore objects in the input visual stimulus, modifying locations of atleast one of the one or more objects in the input visual stimulus, andmodifying a size of at least one of the one or more objects in the inputvisual stimulus.

In some embodiments, the color-emotion analysis tool 118 of theinformation density toolkit system 110 is used to analyze an input imageor other visual stimulus. Color can influence an individual's emotionsand decision-making. The associations between color and emotion areuniversal, and can be measured in the brain. Furthermore, color's impacton emotion has been shown to have a significant impact on anindividual's cognitive processes, affecting performance and recall.Therefore, measures of color and associated emotions can provide auseful input in various other tools of the information density toolkitsystem 110, including but not limited to the emotional resonance scoringtool 120.

As used herein, emotion refers to a set of discrete reactions to aninternal or external event, and may represent a complex reaction patterninvolving experiential, behavioral and physiological elements. Emotionaffects users, where the affect may be an experience of feeling oremotion, ranging from suffering to elation, and from the simplest to themost complex sensations of feeling, and from the most normal to the mostpathological emotional reactions. Often described in terms of positiveaffect or negative affect, both mood and emotion are consideredaffective states. Along with cognition and conation, affect is one ofthe three traditionally identified components of the mind. Emotionalvalence is the value associated with a stimulus, as expressed on acontinuum from pleasant to unpleasant, or from attractive to aversive.In factor analysis and multidimensional scaling studies, emotionalvalence is one of two axes (or dimensions) on which an emotion can belocated, the other axis being arousal (expressed as a continuum fromhigh to low). For example, happiness is typically characterized bypleasant valence and relatively high arousal, whereas sadness ordepression is typically characterized by unpleasant valence andrelatively low arousal.

FIG. 15 shows a system flow 1500 for the color-emotion analysis tool118. The system flow 1500 starts with an input image 1501 andinformation density output 1503 (e.g., from the information densitymapping tool 112), which are provided as input for color analytics 1505.The color analytics 1505 performs a color analysis on any identifiedobjects or user-specified areas of interest in the input image 1501. Foreach identified object, the color analytics 1505 provides that object'scolor-emotion categorization 1507 as well as a color-emotion score 1509(e.g., a score based on the Geneva Emotion Wheel).

An exemplary process for color-emotion analysis of a visual stimuluswill now be described in more detail with reference to the flow diagramof FIG. 16 . It is to be understood that this particular process is onlyan example, and that additional or alternative processes forcolor-emotion analysis of a visual stimulus may be used in otherembodiments.

In this embodiment, the process includes steps 1600 through 1606. Thesesteps are assumed to be performed by the information density toolkitsystem 110 utilizing the color-emotion analysis tool 118 and the designpersonalization tool 124.

The process begins with step 1600, obtaining an information densitymatrix for an input visual stimulus, the information density matrixcharacterizing information density of feature points in the input visualstimulus. The input visual stimulus may comprise an image file. Theimage file may be of a product, an application screen or portionsthereof, a website or webpage or portion thereof, a document, etc., thatis to be analyzed for its associated color-emotion characteristics.

In step 1602, content of the input visual stimulus is analyzed utilizingthe information density matrix to identify one or more objects in theinput visual stimulus. Color-emotion scores for each of the one or moreobjects are determined in step 1604. The color-emotion score for a givenone of the one or more objects characterizes an emotional effect of thegiven object on a user viewing the input visual stimulus based at leastin part on color of the given object. The color-emotion score may have amagnitude indicating an intensity thereof, as well as a sign indicatingwhether the emotional effect is positive or negative. A design of theinput visual stimulus is modified in step 1606 to adjust at least one ofthe determined color-emotion scores for at least one of the one or moreobjects. Step 1606, for example, may include adjusting a color of atleast one of the one or more objects to change a magnitude of thecolor-emotion score of that object or a sign of the color-emotion scoreof that object.

As social beings, the human brain has developed complex emotionalpathways that can prioritize attention and engagement, improve brandrecall and loyalty, and increase a sense of attachment and community.Therefore, the ability to predict an individual's emotional response toa novel design or product would be advantageous. In some embodiments,the emotional resonance scoring tool 120 provides a solution forestimating how likely an object (e.g., a product or webpage component)is to elicit an emotional response from a viewer. This is particularlyuseful as an input for the estimation and mapping of visual attentionand cognitive load (e.g., using the information density mapping tool112, using the cognitive load index generation tool 114, using thebehavioral intent estimation tool 116, using the color-emotion analysistool 118, using the visual attention likelihood estimation tool 122,using the design personalization tool 124, etc.). The emotionalresonance scoring tool 120 represents part of a scalable solution (e.g.,of the information density toolkit system 110) for product designers andresearchers to evaluate the efficacy of their designs prior to launch.The emotional resonance scoring tool 120 provides various advantages inits ability to predict customer engagement and purchase decisions,especially when combined with various other tools offered within theinformation density toolkit system 110.

Emotions can play a large role in the decision-making process. Consumersoften lack introspective access to their emotional states, and it istherefore difficult to measure the impact of emotional content onconsumers' decision-making processes. The neural circuitry underlyingemotional processing is highly integrated with key areas of thedecision-making process. For example, the hippocampus is a brain areawell-known for storing and retrieving memories. Under highly emotionalor arousing circumstances, the amygdala (e.g., a brain area best knownfor its control over our emotional “fight or flight” responses)modulates hippocampal activity and memory storage. Similarly, emotionalcircumstances cause the amygdala to modulate activity in the visualcortex, prioritizing visual attention toward emotionally-salientstimuli.

Studies have demonstrated the effectiveness of emotional resonance oncustomers' willingness to pay, and on product satisfaction. In someexperiments, different participant groups were subliminally presentedwith either happy, sad or neutral faces. The groups then poured,consumed and rated their willingness to pay for a non-alcoholicbeverage. Participants exposed to happy faces poured and consumed moreof the beverage, while also rating it more highly and reporting a higherwillingness to pay. Participants exposed to sad faces demonstrated theopposite effect. Therefore, emotional primes (e.g., happy or sad faces)can bias appraisal and decision-making behavior.

As discussed above, color can also influence an individual's emotionsand decision making. The associations between color and emotion areuniversal, and can be measured in the brain. Furthermore, color's impacton emotion has been shown to have a significant impact on anindividual's cognitive processes, affecting performance and recall.Therefore, it is important to incorporate measures of color in anypotential measure of emotional resonance.

Given that emotional resonance can meaningfully impact behavioraloutcomes, it would be beneficial to be able to measure the emotionalcontent of a product or design. To this end, the emotional resonancescoring tool 120 provides a solution for an automated method ofmeasuring predicted emotional responses to content through emotionalresonance scores. The solutions described herein allow for themeasurement of all visual content, including but not limited to text,image and video-based content. Researchers and designers will be able touse the emotional resonance scores to predict and enhance viewers'emotional responses.

There is currently no automated methodology for measuring the predictedemotional response of a viewer to content (e.g., product or webdesigns). Since emotional responses can have a large impact on thebrain's appraisal and decision-making processes, it is useful to capturethis information, especially as a component of more complex processes(e.g., visual attention, willingness to pay, product satisfaction,etc.).

Conventional approaches lack the ability to quantitatively measure andpredict a viewer's emotional response to content. Although the use ofhappy faces has a long history in marketing practices, no methodologyhas been developed to measure and predict the effectiveness of thisemotional content. Without appropriate measures, it is difficult tooptimize emotional content. Conventional approaches also lack theability to comprehensively measure the emotional resonance of visualcontent as a whole. Although some vendors provide emotional analysis of,for example, text through natural language processing (NLP) algorithms,there are currently no solutions that can offer a comprehensiveemotional resonance score based on all types of content (e.g., acombined score for static images, moving images, colors, and textualcomponents of a web-page or product).

The emotional resonance scoring tool 120 can operate as a component of asuite of analytic tools provided by the information density toolkitsystem 110, and has a primary function of measuring and predictingemotional resonance to a visual stimulus. The emotional resonance scoretakes input from various other tools in the information density toolkitsystem 110 (e.g., such as the information density mapping tool 112, thecognitive load index generation tool 114, the behavioral intentestimation tool 116, etc.), and returns values estimating the visualstimulus' likelihood to elicit an emotional response from its viewers.This score takes into account all aspects of the visual stimulus (e.g.,text, colors, static images, and moving images) to provide acomprehensive score, which may be used in various other tools in theinformation density toolkit system 110 (e.g., such as the visualattention likelihood estimation tool 122, the design personalizationtool 124, etc.).

In some embodiments, the emotional resonance scoring tool 120 combinesthe output of a variety of algorithms, where each individual algorithmor approach is selected to assess a specific type of visual stimulus.Various different possible types of visual stimuli can be utilized,including but not limited to: text-based content (e.g., includingfont-based emojis); color content; static images; moving images orvideos; manual user input; etc. Each identified object is analyzedaccordingly and given an emotional resonance score. The emotionalresonance score may range from −100 to 100, with negative scoresrepresenting negatively-valenced emotions and positive scoresrepresenting positively-valenced emotions, and with higher magnitudescores representing greater emotional intensity and lower magnitudescores representing greater emotional intensity.

FIG. 17 shows a system flow 1700 for the emotional resonance scoringtool 120. The system flow 1700 receives an input image or other visualstimulus 1701 (e.g., entered by a user into the information densitytoolkit system 110). The input image 1701, in some embodiments, istagged with or includes optional user-specified AOIs. Informationdensity output or mapping 1703 is also received, which includes spatialclustering output from the information density mapping tool 112. Objectdetection 1705 is then performed to determine stimuli categorization1707. The object detection 1705 may include optional specifications andalgorithm selection to recognize objects and perform semanticsegmentation. The object detection 1705 uses the input image 1701 andinformation density mapping 1703 to detect, recognize and classifyobjects within the image into one of a set of different types ofstimuli, such as: text-based stimuli; static image stimuli; moving image(e.g., movies, animations, etc.) stimuli; optional user input stimuli;etc., output as the stimuli categorization 1707.

Color analysis 1709 is then performed on any identified object and/orany user-specified AOIs. In order to facilitate this, the color-emotionanalysis tool 118 may be called. For each identified object, thecolor-emotion analysis tool 118 may provide a color emotion score (e.g.,based on the Geneva Emotion Wheel).

Text-based analysis 1711 (e.g., NLP) is performed for any identifiedtext-based objects. Text-based analysis 1711 may include performing asentiment analysis using various NLP methodologies. The algorithm maybegin with extracting the text data, and then evaluating the extractedtext data with a sentiment lexicon (e.g., the Valence Aware Dictionary)and a sentiment classifier (e.g., the Valence Aware Dictionary andsEntiment Reasoner (VADER) algorithm). The text-based analysis 1711 willthen produce an emotional resonance score for each identified textobject. It should be noted that the text-based analysis 1711 can beperformed on any language, with the precondition that an existingsentiment lexicon for that language can be identified.

Static image analysis 1713 includes analysis of any identified staticimage objects, by training a semantic segmentation network using deeplearning. Each individual pixel may be classified, which results in anoutput image segmented by class. A default list of classes (e.g., faces,laptop, smile, etc.) is pre-trained and “built-in” to the emotionalresonance scoring tool 120. This list can be updated by uploadingappropriate image files for training (e.g., an additional class labeled“keyboard” can be added by uploading a training set of images ofkeyboards). The identified classes of image objects are then scored foremotional content.

Moving image analysis 1715 includes analysis of any identified movingimages (e.g., movie files, files that contain animated content, etc.).The user can specify this content for analysis, and requires inputting aseparate file that contains only the movie. The movie file willsubsequently be broken down into stills. The emotional resonance scoringtool 120 will automatically detect the content within the original inputby comparing any identified static images to user-inputted movie stills.The movie will then be analyzed in the same fashion as static imagesduring static image analysis 1713. The emotional content of each staticimage will be scored and a combined average will be provided for themovie object as a whole.

Optional user input 1717 is then processed. The user may optionallyinput data to modify the analysis. For example, a user may hypothesizethat an individual's recent customer satisfaction score might affect hisor her emotional experience. The user must then input a set ofcomponents such as: a score; the range of possible scores; and theweight that should be applied to this input relative to the otheridentified stimuli. The optional user input 1717 can be any hypothesizedmodifier, as long as the three aforementioned components are satisfied.

Combined analytics 1719 are then performed, which may includedetermining relative size, preferential access, stimuli quantity andpotential impact. A weighted average 1721 may be output by the combinedanalytics 1719 once an emotional resonance score has been attached toeach identified object. In some embodiments, two types of weights areconsidered: analysis type (e.g., color vs. text-based analysis); andrelative object size. These weights are combined with individual objectscores, and scaled between −100 and 100 to calculate an overallemotional resonance score. The final output of the system flow 1700 isan emotional resonance score 1723, which is a numeric output (e.g., in arange between −100 and 100). Negative numbers indicate negative valence(e.g., sad), while positive numbers indicate positive valence (e.g.,happy). Low magnitude numbers indicate low intensity (e.g., content),whereas high magnitude numbers indicate high intensity (e.g., ecstatic).This number can be displayed independently to the user and/or used asinput to other tools within the information density toolkit system 110.

Conventional approaches for capturing the emotional resonance of contentare limited to A/B testing or user testing. While such conventionalapproaches may be useful, they lack the ability for prediction andscaling. The automated approach provided by the emotional resonancescoring tool 120 described herein directly addresses these and otherdrawbacks of conventional approaches. The emotional resonance scoringtool 120 represents a novel approach for measuring and predicting aviewer's emotional response to visual content. By providing a numericemotional resonance score, users are given the ability to directlycompare content (e.g., two prototypes of a product) and contrast thepredicted effectiveness of their designs at eliciting an emotionalreaction. For example, if version 1.0 received a score of −45 andversion 2.0 received a score of 30, one can conclude that there was a 75point increase from negative to positive emotional content, but a 15point decrease in the intensity of the emotional content from version1.0 to 2.0. The emotional resonance scoring tool 120 can alsoautomatically identify and classify different types of content, and thenmeasure those separate components based on established, scientificknowledge of how humans perceive various types of stimuli. The emotionalresonance scoring tool 120 then combines individual component scoresinto a single comprehensive and interpretable emotional resonance score.The emotional resonance scoring tool 120 therefore provides acomprehensive solution, rather than conventional approaches which aretypically limited to analyzing a single type of data (e.g., only text,or only static images).

While the connections between consumers' emotional states andconsumption patterns are well documented, conventional approaches lackthe ability to measure content's emotional resonance and use thatquantification to predict consumptions patterns. The emotional resonancescoring tool 120 bridges this gap, allowing for a direct, measurablerelationship between content's capacity for eliciting emotionalresponses and consumptive behavior. This is true of any visual stimulus,ranging from products to web design. Moreover, the emotional resonancescoring tool 120 may be used as part of a larger suite of analytic toolsprovided by the information density toolkit system 110, addressing anoften overlooked, yet highly impactful, element of decision making:consumer emotions. By quantifying a typically nebulous anddifficult-to-constrain concept, the automated emotional resonancescoring tool 120 offers a scalable solution to measurably improve designanalytics, customer experience, brand loyalty and conversion rates.

FIG. 18 shows an example 1800 of an input visual stimulus (e.g., awebsite homepage) overlayed with a grid. In this example, the user haschosen three grid squares 1801, 1803 and 1805 as areas of interest. Theemotional resonance scoring tool 120 evaluates each of the grid squares1801, 1803 and 1805 for emotional content (e.g., emotional imagery,text, or color associations). The probability that the content withineach square will elicit an emotional response from a viewer isrepresented here with colors (Light Grey: High; White: Medium; DarkGrey: Low). It is important to note that the output represents thelikelihood of eliciting any emotion (e.g., positive or negative) andshould not be interpreted as the likelihood of eliciting a positive ornegative emotion.

An exemplary process for emotional resonance scoring of a visualstimulus will now be described in more detail with reference to the flowdiagram of FIG. 19 . It is to be understood that this particular processis only an example, and that additional or alternative processes foremotional resonance scoring of a visual stimulus may be used in otherembodiments.

In this embodiment, the process includes steps 1900 through 1906. Thesesteps are assumed to be performed by the information density toolkitsystem 110 utilizing the emotional resonance scoring tool 120 and thedesign personalization tool 124.

The process begins with step 1900, obtaining an information densitymatrix for an input visual stimulus, the information density matrixcharacterizing information density of feature points in the input visualstimulus. The input visual stimulus may comprise an image file. Theimage file may be of a product, an application screen or portionsthereof, a website or webpage or portion thereof, a document, etc., thatis to be analyzed for its associated emotional resonance.

In step 1902, content of the input visual stimulus is analyzed utilizingthe information density matrix to identify one or more objects in theinput visual stimulus. Step 1902 may include identifying objects in oneor more user-specified areas of interest within the input visualstimulus. Emotional resonance scores associated with each of the one ormore objects in the input visual stimulus are determined in step 1904. Agiven emotional resonance score for a given one of the one or moreobjects characterizes an intensity and an emotional valence of anemotional effect of the given object on a user viewing the input visualstimulus. Step 1904 may comprise performing color-emotion analysis togenerate a color-emotion score characterizing the emotional effect ofthe given object on the user viewing the input visual stimulus based atleast in part on one or more colors in the given object.

In some embodiments, determining the emotional resonance scores for theone or more objects comprises classifying stimuli type of the one ormore objects, the stimuli type comprising at least one of text-basedstimuli, static image stimuli, and moving image stimuli. The FIG. 19process may further include determining an overall emotional resonancescore for the input visual stimulus as a weighted average of theemotional resonance scores determined for the one or more objects,wherein weights are assigned to the one or more objects based at leastin part on the classified stimuli type of the one or more objects andrelative object size of the one or more objects. Determining the overallemotional resonance score for the input visual stimulus may be furtherbased at least in part on a user-input emotional resonance scorecomponent for the user viewing the input visual stimulus, the user-inputemotional resonance score component characterizing external factorsaffecting emotion of the given user at least one of prior to and duringviewing of the input visual stimulus. The overall emotional resonancescore for the input visual stimulus may have a magnitude characterizingan overall intensity of an emotional effect of the input visual stimuluson the user and a sign characterizing an overall emotional valence ofthe emotional effect of the input visual stimulus on the user.

The given object may comprise text content, and determining the givenemotional resonance score for the given object may comprise extractingthe text content from the given object and performing sentiment analysisfor the text content using a machine learning-based sentimentclassifier. The given object may also or alternatively comprise one ormore static images, and determining the given emotional resonance scorefor the given object may comprise utilizing a machine learning-basedsemantic segmentation network that identifies, for each of one or moreportions of the one or more static images, one of a set of semanticclasses, and generating the given emotional resonance score based atleast in part on the identified semantic classes for the one or morestatic images. The given object may also or alternatively comprise oneor more moving images, and determining the given emotional resonancescore for the given object may comprise breaking down the one or moremoving images into two or more static images, generating an emotionalresonance score component for each of the two or more static images, anddetermining the given emotional resonance score for the given objectbased at least in part on an average of the emotional resonance scorecomponents for the two or more static images.

In some embodiments, the FIG. 19 process further comprises generating avisualization comprising a grid of equally spaced bins overlaying theinput visual stimulus. Each of the bins may be associated with aprobability that content within that bin will elicit an emotionalresponse from the user viewing the input visual stimulus, theprobability being determined based at least in part on the emotionalresonance scores determined for the one or more objects.

The FIG. 19 process continues with step 1906, modifying a design of theinput visual stimulus to adjust at least one of the emotional resonancescores associated with at least one of the one or more objects in theinput visual stimulus. Step 1906 may comprise modifying at least one ofthe one or more objects to adjust its emotional resonance score relativeto the emotional resonance scores of other ones of the one or moreobjects in the input visual stimulus. Step 1906 may also oralternatively comprise modifying information content of at least one ofthe one or more objects in the input visual stimulus and modifying asize of at least one of the one or more objects in the input visualstimulus.

Visual attention is often preceded by a pause in eye-movement on an areaof interest, which is referred to as a “fixation.” A fixation refers tothe orientation of the eyes so that the image of a viewed object fallson each fovea, in the central part of the retina. Visual attentionrefers to the process by which one item (e.g., a target) is selected foranalysis from among several computing items (e.g., distractors).Research demonstrates that eyes selectively fixate on areas that are themost informative, and that cognitive factors such as behavioral intentcan have a substantial impact on the spatial and temporal dynamics ofthese fixation patterns. Given that the average time consumers spendinspecting a visual stimulus (e.g., a novel product or webpage) can bemeasured in seconds, it is important for designers to be able toprioritize a user's visual attention to a desired area (e.g., to a CTAbutton, to a new product feature, etc.). There is currently no automatedmethod of using top-down cognitive factors (e.g., cognitive load,emotional resonance, and behavioral intent) and behavioral biases (e.g.,horizontal and vertical visual biases) to predict whether or not a userwill attend to an object or area of interest. In tandem with other toolsof the information density toolkit system 110 (e.g., the informationdensity mapping tool 112, the cognitive load index generation tool 114,the behavioral intent estimation tool 116, the color-emotion analysistool 118, the emotional resonance scoring tool 120, and the designpersonalization tool 124), the visual attention likelihood estimationtool 122 offers researchers and designers a means of predicting the mostlikely locations within a visual stimulus to draw a user's attention.The visual attention likelihood estimation tool 122 can be used toimprove digital content and design, which should lead to increases indirect (e.g., conversion rates, invention licensing, etc.) and indirect(e.g., CLTV, engagement, usability, etc.) revenue.

When individuals view products, webpages or other visual stimulus, theireyes will quickly move from one area of interest to another, stoppingbriefly at certain points on the screen to gather information. Thesetargeted pauses in eye-movements are called “fixations.” FIGS. 20A and20B show examples 2000 and 2005 of fixations as a user views differentvisual stimulus. The brain takes a “snapshot” at each fixation point andthen stitches together the bits of information from each fixation togenerate a higher resolution mental picture of the visual environment.The example 2000 of FIG. 20A shows an eye-tracking fixation pattern foran image of a face, with the labeled numbers illustrating an examplefixation sequence that begins at the woman's left eye (1), and the movesto the right eye (2), the mouth (3), the nose (4), back to the eyes (5)and (6), and finally to the forehead (7) and hairline (8). The example2005 of FIG. 20B shows an eye-tracking fixation pattern for text-basedreading. The circles indicate fixation points where the eye stops,commonly skipping words or portions of words. The brain combines thefixations as one moves along to complete the sentence cognitivelyafterwards. This cognitive stitching process allows readers to easilycomprehend misspelled words—often completely subconsciously.

In humans, the center of the visual field is not only associated withthe highest visual acuity, but also with a greater number of neuronsdedicated to the processing of that information relative to theperiphery. According to neuroscience research, a central task of theattentional system of the human brain is controlling gaze, along withretrieving and filtering relevant features from the environment.Therefore, determining where, when, and how long to fixate our eyes onan object are small, but very important decisions for our brains to makeat any given moment. Even seemingly insignificant changes to a fixationpattern can completely alter how an individual perceives an environment.The importance of visual attention is exemplified by a phenomenon called“inattentional blindness,” in which an object can be clearly visible,but not “seen.” Unattended objects cannot be perceived or laterrecalled, because the brain has filtered out and discarded thatinformation in order to preferentially allocate cognitive resources toattended objects. In other words, if an individual's eyes and subsequentvisual attention have not been focused on a particular object, the brainwill not perceive that object, nor will that individual be able torecall that object in the future.

Bottom-up information processing is information processing in whichincoming stimulus data initiate and determine the higher-level processesinvolved in their recognition, interpretation, and categorization. Forexample, in vision, features would be combined into objects, and objectsinto scenes, recognition of which would be based only on the informationin the stimulus input. Typically, perceptual or cognitive mechanisms usebottom-up processing when information is unfamiliar or highly complex.Top-down information processing is information processing in which anoverall hypothesis about or general conceptualization of a stimulus isapplied to and influences the analysis of incoming stimulus data. Forexample, in reading, knowledge about letter and word frequencies,syntax, and other regularities in language guides recognition ofincoming information. In this type of processing, a person'shigher-level knowledge, concepts, or expectations influence theprocessing of lower level information (see proofreader's illusion).Typically, perceptual or cognitive mechanisms use top-down processingwhen information is familiar and not especially complex.

Eye movements reflect an individual's search for information, and thisprocess is simultaneously influenced by both “bottom-up” stimuli-drivenfactors as well as “top-down” cognitive factors. Bottom-up factorsrepresent low-level visual factors that can draw the eye, such as color,luminance, and movement. There are also bottom-up behavioral biases,such as vertical and horizontal visual biases that reflect our tendencyto inspect scenes from left-to-right and top-to-bottom. Top-downfactors, such as cognitive load, emotional resonance, and behavioralintent, also dictate the way in which eyes move. For example, researchdemonstrates that as task difficulty and cognitive load increase,individuals will fixate more often and for a longer duration in order tofully comprehend the task at hand.

Similarly, behavioral intent can categorically change the spatial andtemporal patterns of eye-movements, leading individuals to fixate ontask-relevant objects, while ignoring more visually salient (buttask-irrelevant) stimuli. The affective nature of a stimulus can alsohave a substantial impact in how an individual explores a visualenvironment, with emotional stimuli often biasing attention. This isespecially true in social contexts, wherein research shows that gazeallocation is more often directed at facial areas that express emotions(e.g., someone's eyes and mouth) relative to other more visually salientareas (e.g., a shiny earring). Thus, cognitive load and intentfundamentally change the way an individual moves their eyes whencompleting a visual task.

It is also important to note that eye fixation on a particular objectactually biases subsequent choice of that object in a decisionenvironment. This means that, when a consumer has spent more timelooking at object A compared to object B, that person will be morelikely to purchase object A over object B. This research underscores theutility of accurate predictions of consumers' visual attention. FIG. 21shows a plot 2100, illustrating the correlation between fixation timeand likelihood of purchase. As total fixation time on a productincreases, the probability of purchasing that product significantlyincreases.

Given this background, it is evident why it is useful to be able topredict where an individual might focus his or her attention whenviewing a visual stimulus (e.g., a product, a webpage, etc.) to ensurethat desired areas of interest are engaged with as intended by thedesigner, and to predict the likelihood of a particular design to bias aviewer's choice behavior. In addition to conventional research-basedeye-tracking methods that offer post-hoc analyses of visual content,there are conventional approaches (e.g., computer algorithms) that havebeen developed to predict eye movements based on low-level bottom-upfeatures within visual scenes. However, such conventional approaches donot incorporate important behavioral biases and top-down cognitivefactors that can bias visual attention, such as: vertical and horizontalbiases; viewer intent; a viewer's cognitive load; and the emotionalsalience of stimuli.

An individual's behavioral intent, along with the cognitive demands andemotional salience of the visual environment, are important cognitivefactors that determine where that individual's visual attention will beplaced at any given moment. However, designers and researchers do notcurrently have a method of using cognitive factors to predict the mostlikely locations of visual attention when viewing an input stimulus(e.g., an object, a product, a website, etc.). Conventional approachesfor eye-movement analysis are typically limited to research settingsusing in-lab or remote (e.g., web-based) eye-tracking software and/orhardware. While some conventional approaches provide automatedalgorithms to predict eye-movements, such predictions are typicallybased on low-level visual salience features (e.g., color, luminance,etc.) with little or no incorporation of top-down cognitive factors. Apredictive, automated method of estimating likely fixation patternsbased on behavioral intent, cognitive load, and emotional resonancewould be of great value as a scalable means of evaluating a design'spotential effectiveness. By predicting the likelihood of visualattention to fall within one area (e.g., a high value product) over asecond area (e.g., a low value product), one can optimize designs (e.g.,products, website layouts, product assortments, etc.) to maximizetargeted attention by its viewers. Therefore, estimations of the mostlikely areas of a design to elicit visual attention can act as a measureof that design's effectiveness in transmitting information. This isparticularly relevant in decision-making contexts, as fixation time onone option over another option will bias an individual's choice towardsthe option associated with the longest fixation time.

A design's effectiveness relies on its ability to transmit the intendedinformation to its viewer. Likewise, visual perception and informationprocessing rely on eye fixations to areas of high information. Visualenvironments and tasks that place high cognitive demands on anindividual viewer will limit the way that individual's visual attentionis allocated, thereby limiting the potential effectiveness of a design.Presently, there is no methodology that can estimate the most likelyareas of visual attention based on the estimated cognitive load ofviewers. Conventional approaches either rely on eye-tracking in the labor with webcams (which cannot be automated, offer advanced predictions,or be applied at scale), or on computer algorithms that are lessaccurate due to their lack of incorporation of the effects of cognitiveload on visual attention. The cognitive load index generation tool 114,described above, provides a solution for measuring cognitive load ofvisual stimulus.

The human brain uses visual attention to allocate cognitive resources torelevant aspects of an environment when completing a task, effectivelyfiltering out most task-irrelevant information. Behavioral intent (e.g.,browsing vs. searching) determines which visual features are relevant ornot at any given moment. Therefore, behavioral intent plays a vastlyimportant role in dictating which features within a visual environmentare most likely to elicit visual attention. While methodologies havebeen developed to predict behavioral intent from eye movement patterns,predicting eye movement patterns based on intent has yet to beaccomplished, likely due to the covert nature of behavioral intent andthe subsequent difficulty in predicting and measuring that intent. Thebehavioral intent estimation tool 116, described above, provides asolution for predicting behavioral intent given an input visualstimulus.

Due to our social nature, the human brain has evolved to prioritizevisual content that conveys emotion, especially regarding faces and bodyposture. Visual attention is largely biased towards emotion-conveyingareas of a face, irrespective of other areas that are more visuallysalient. Irrespective of whether the stimulus is social (e.g., a face)or non-social (e.g., text), the emotion being conveyed by that stimulustypically elicits individual approach and avoidance behaviors.Therefore, the emotional resonance of an object or feature can biasvisual attention either toward (e.g., approach behavior) or away (e.g.,avoidance behavior) from it. This suggests that any algorithm predictingeye movement patterns should take into consideration not just low-levelfeatures of visual salience, but also the emotional content of thevisual environment. However, conventional approaches fail to capture theway in which emotional content elicits visual attention. Thecolor-emotion analysis tool 118 and the emotional resonance scoring tool120 provide a solution for determining the emotional resonance of aninput visual stimulus.

Humans, in general, have a left-right horizontal viewing bias. Althoughthis bias can be lessened by cultural factors (e.g., in cultures wherethe written language is read from right to left), it still represents astrong behavioral bias during visual exploration. A left-to-right biashas even been observed in other species, including monkeys and dogs.There also exists a similar vertical bias towards the top of one'svisual field (e.g., top of the screen) or the “top” of a representationor object (e.g., the head of a figure). As a result of these horizontaland vertical biases, the diagonal from the top-left to the bottom-rightof a visual field represents the optimal optical placement for naturallyattracting visual attention.

The visual attention likelihood estimation tool 122 takes input from avisual stimulus, and from various other tools of the information densitytoolkit system 110 (e.g., the information density mapping tool 112, thecognitive load index generation tool 114, the behavioral intentestimation tool 116, the color-emotion analysis tool 118, and theemotional resonance scoring tool 120), and returns an estimation of thelikelihood that one or more AOIs will elicit visual attention. If theuser does not specify any AOIs, then the one or more clusters identifiedby the information density mapping tool 112 can serve as default AOIs.

FIG. 22 shows a system flow 2200 for operation of the visual attentionlikelihood estimation tool 122. The visual attention likelihoodestimation tool 122 takes as input a visual stimulus or image 2201, aswell as an information density output 2203 (e.g., from the informationdensity mapping tool 112), which are input to path detection 2205resulting in cognitive load index output 2207 (e.g., from the cognitiveload index generation tool 114). The path detection 2205 may includeoptional specifications and selection of algorithms, as well as objectrecognition and semantic segmentation. Optical centering analysis 2209is then performed, followed by behavioral intent estimation 2211 (e.g.,from the behavioral intent estimation tool 116, or from manual input)and emotional resonance scoring 2213 (e.g., from the emotional resonancescoring tool 120). The user also has the option of including additionaloptional user input 2215 (e.g., measures and their associated weightsvia manual input). The various cognitive measures are then provided tocombined analytics 2217. The combined analytics 2217 can scale, weightand combine the various measures (e.g., information density output 2203,the cognitive load index output 2207, the optical centering analysis2209, the behavioral intent estimation 2211, the emotional resonancescore 2213 and optional user input 2215) into one measure withinuser-specified AOIs to product the weighted average 2219. The weightedaverage 2219 will be used as part of combined analytics 2217 to generatean output measure referred to as a visual attention likelihoodestimation 2221 (e.g., in the form of a percentage, with a range from0-100%).

Conventional approaches for predicting visual attention primarily relyon low level visual cues (e.g., contrast, luminance, etc.). Somealgorithms take into consideration more higher-level cognitive factors,such as whether the visual stimulus contains a face. However, theconventional approaches do not consider high-level factors such ascognitive load, emotional resonance, behavioral intent, and opticalcentering biases that are considered by the visual attention likelihoodestimation tool 122.

The visual attention likelihood estimation tool 122 provides a number ofadvantages, including in the consideration of cognitive load, emotionalresonance, intent relevance, and optical centering components. Thein-moment cognitive demands placed on individuals constrains the spatialallocation of visual attention. The visual attention likelihoodestimation tool 122 utilizes this insight by incorporating aquantitative measure of cognitive load (e.g., provided via the cognitiveload index generation tool 114) into the visual attention likelihoodprediction. An individual's intent has a profound impact on what he orshe sees and attends to within a visual environment. The visualattention likelihood estimation tool 122 takes advantage of this insightby taking behavioral intent (e.g., provided via the behavioral intentestimation tool 116) as input into the visual attention likelihoodprediction. The emotional content of a visual environment can affect anindividual's eye-movements and visual attention. The visual attentionlikelihood estimation tool 122 capitalizes on this insight byincorporating a quantitative measure of emotional resonance (e.g.,provided via the color-emotion analysis tool 118 and the emotionalresonance scoring tool 120) into the visual attention likelihoodprediction.

Due to the evolutionary vertical and horizontal visual biases, humans'optical center is not the same as true center. Viewing patterns tend tobe biased from left-to-right and top-to-bottom. The visual attentionlikelihood estimation tool 122 utilizes such insights in order todeliver better predictions of visual attention likelihood at any givenspatial coordinate.

FIG. 23 shows an example 2300 of visual attention likelihood estimationsfor three AOIs 2301, 2303 and 2305 on a website homepage. Here,behavioral intent is representative of a user seeking technicalspecifications post-purchase. The AOI 2301 receives a 43.8% likelihood.The AOI 2301 ranks: high on optical centering (e.g., it bisects thecenter line high up); high on cognitive load (e.g., it has highinformation content); medium on emotional resonance (e.g., via itsemotional word choices); and low on intent (e.g., little relevantinformation). The AOI 2303 in the bottom center receives a 56.3%likelihood of receiving visual attention. The AOI 2303 ranks: middle-lowon optical centering (e.g., it is below the center line); low oncognitive load (e.g., it has little information content); low onemotional resonance (e.g., it has no emotional information); and high onintent (e.g., it has highly relevant information). The AOI 2305 in thetop right receives a 68.8% likelihood of receiving visual attention. TheAOI 2305 ranks: medium on optical centering (e.g., it is just abovecenter line); medium on cognitive load (e.g., it has medium informationcontent); high on emotional resonance (e.g., via emotion conveyedthrough faces); and low on intent (e.g., it has little relevantinformation).

An exemplary process for visual attention likelihood estimation forobjects of a visual stimulus will now be described in more detail withreference to the flow diagram of FIG. 24 . It is to be understood thatthis particular process is only an example, and that additional oralternative processes for visual attention likelihood estimation forobjects of a visual stimulus may be used in other embodiments.

In this embodiment, the process includes steps 2400 through 2408. Thesesteps are assumed to be performed by the information density toolkitsystem 110 utilizing the visual attention likelihood estimation tool 122and the design personalization tool 124.

The process begins with step 2400, obtaining an information densitymatrix for an input visual stimulus, the information density matrixcharacterizing information density of feature points in the input visualstimulus. The input visual stimulus may comprise an image file. Theimage file may be of a product, an application screen or portionsthereof, a web site or webpage or portion thereof, a document, etc.,that is to be analyzed for its associated visual attention likelihoodestimations.

In step 2402, content of the input visual stimulus is analyzed utilizingthe information density matrix to identify one or more objects in theinput visual stimulus. The one or more objects may comprise areas ofinterest corresponding to clusters of feature points in the informationdensity matrix. At least one of the one or more objects may comprise auser-specified area of interest.

Two or more cognitive measures are evaluated in step 2404 for each ofthe one or more objects in the input visual stimulus. The two or morecognitive measures for a given one of the one or more objects maycomprise a cognitive load index measure, an optical centering measure, abehavioral intent estimation measure, an emotional resonance measure,and one or more user-input cognitive measures. The cognitive load indexmeasure characterizes cognitive energy required to mentally process thegiven object in the input visual stimulus. The optical centering measurecharacterizes a visual attention bias of the given object based at leastin part on its location within the input visual stimulus. The visualattention bias of the given object may be based at least in part on adistance of the given object from a diagonal extending from a top-leftcorner of the input visual stimulus to a bottom-right corner of theinput visual stimulus. The visual attention bias of the given object mayalso or alternatively be based at least in part on a distance from a topof the input visual stimulus. The visual attention bias of the givenobject may further or alternatively be based at least in part on adistance from a left-hand side of the input visual stimulus. Thebehavioral intent estimation measure characterizes estimatedprobabilities of two or more different behavioral intents of the userviewing the given object in the input visual stimulus. The emotionalresonance measure characterizes an intensity and an emotional valence ofan emotional effect of the given object on the user viewing the inputvisual stimulus.

In step 2406, a likelihood of a user viewing the input visual stimulusto fixate attention on respective ones of the one or more objects in theinput visual stimulus is estimated based at least in part on the two ormore cognitive measures of each of the one or more objects in the inputvisual stimulus. Step 2406 may comprise, for a given one of the one ormore objects, computing a weighted average of the two or more cognitivemeasures of the given object. The FIG. 24 process may continue with step2408, modifying a design of the input visual stimulus to adjust theestimated likelihood of the user viewing the input visual stimulus tofixate attention on at least one of the one or more objects in the inputvisual stimulus. Step 2408 may comprise at least one of modifyinginformation content of said at least one of the one or more objects inthe input visual stimulus and adjusting a location of said at least oneof the one or more objects in the input visual stimulus.

It is to be appreciated that the particular advantages described aboveand elsewhere herein are associated with particular illustrativeembodiments and need not be present in other embodiments. Also, theparticular types of information processing system features andfunctionality as illustrated in the drawings and described above areexemplary only, and numerous other arrangements may be used in otherembodiments.

Illustrative embodiments of processing platforms utilized to implementfunctionality for information density processing and analysis will nowbe described in greater detail with reference to FIGS. 25 and 26 .Although described in the context of system 100, these platforms mayalso be used to implement at least portions of other informationprocessing systems in other embodiments.

FIG. 25 shows an example processing platform comprising cloudinfrastructure 2500. The cloud infrastructure 2500 comprises acombination of physical and virtual processing resources that may beutilized to implement at least a portion of the information processingsystem 100 in FIG. 1 . The cloud infrastructure 2500 comprises multipleVMs and/or container sets 2502-1, 2502-2, . . . 2502-L implemented usingvirtualization infrastructure 2504. The virtualization infrastructure2504 runs on physical infrastructure 2505, and illustratively comprisesone or more hypervisors and/or operating system level virtualizationinfrastructure. The operating system level virtualization infrastructureillustratively comprises kernel control groups of a Linux operatingsystem or other type of operating system.

The cloud infrastructure 2500 further comprises sets of applications2510-1, 2510-2, . . . 2510-L running on respective ones of theVMs/container sets 2502-1, 2502-2, . . . 2502-L under the control of thevirtualization infrastructure 2504. The VMs/container sets 2502 maycomprise respective VMs, respective sets of one or more containers, orrespective sets of one or more containers running in VMs.

In some implementations of the FIG. 25 embodiment, the VMs/containersets 2502 comprise respective VMs implemented using virtualizationinfrastructure 2504 that comprises at least one hypervisor. A hypervisorplatform may be used to implement a hypervisor within the virtualizationinfrastructure 2504, where the hypervisor platform has an associatedvirtual infrastructure management system. The underlying physicalmachines may comprise one or more distributed processing platforms thatinclude one or more storage systems.

In other implementations of the FIG. 25 embodiment, the VMs/containersets 2502 comprise respective containers implemented usingvirtualization infrastructure 2504 that provides operating system levelvirtualization functionality, such as support for Docker containersrunning on bare metal hosts, or Docker containers running on VMs. Thecontainers are illustratively implemented using respective kernelcontrol groups of the operating system.

As is apparent from the above, one or more of the processing modules orother components of system 100 may each run on a computer, server,storage device or other processing platform element. A given suchelement may be viewed as an example of what is more generally referredto herein as a “processing device.” The cloud infrastructure 2500 shownin FIG. 25 may represent at least a portion of one processing platform.Another example of such a processing platform is processing platform2600 shown in FIG. 26 .

The processing platform 2600 in this embodiment comprises a portion ofsystem 100 and includes a plurality of processing devices, denoted2602-1, 2602-2, 2602-3, . . . 2602-K, which communicate with one anotherover a network 2604.

The network 2604 may comprise any type of network, including by way ofexample a global computer network such as the Internet, a WAN, a LAN, asatellite network, a telephone or cable network, a cellular network, awireless network such as a WiFi or WiMAX network, or various portions orcombinations of these and other types of networks.

The processing device 2602-1 in the processing platform 2600 comprises aprocessor 2610 coupled to a memory 2612.

The processor 2610 may comprise a microprocessor, a microcontroller, anapplication-specific integrated circuit (ASIC), a field-programmablegate array (FPGA), a central processing unit (CPU), a graphicalprocessing unit (GPU), a tensor processing unit (TPU), a videoprocessing unit (VPU) or other type of processing circuitry, as well asportions or combinations of such circuitry elements.

The memory 2612 may comprise random access memory (RAM), read-onlymemory (ROM), flash memory or other types of memory, in any combination.The memory 2612 and other memories disclosed herein should be viewed asillustrative examples of what are more generally referred to as“processor-readable storage media” storing executable program code ofone or more software programs.

Articles of manufacture comprising such processor-readable storage mediaare considered illustrative embodiments. A given such article ofmanufacture may comprise, for example, a storage array, a storage diskor an integrated circuit containing RAM, ROM, flash memory or otherelectronic memory, or any of a wide variety of other types of computerprogram products. The term “article of manufacture” as used hereinshould be understood to exclude transitory, propagating signals.Numerous other types of computer program products comprisingprocessor-readable storage media can be used.

Also included in the processing device 2602-1 is network interfacecircuitry 2614, which is used to interface the processing device withthe network 2604 and other system components, and may compriseconventional transceivers.

The other processing devices 2602 of the processing platform 2600 areassumed to be configured in a manner similar to that shown forprocessing device 2602-1 in the figure.

Again, the particular processing platform 2600 shown in the figure ispresented by way of example only, and system 100 may include additionalor alternative processing platforms, as well as numerous distinctprocessing platforms in any combination, with each such platformcomprising one or more computers, servers, storage devices or otherprocessing devices.

For example, other processing platforms used to implement illustrativeembodiments can comprise converged infrastructure.

It should therefore be understood that in other embodiments differentarrangements of additional or alternative elements may be used. At leasta subset of these elements may be collectively implemented on a commonprocessing platform, or each such element may be implemented on aseparate processing platform.

As indicated previously, components of an information processing systemas disclosed herein can be implemented at least in part in the form ofone or more software programs stored in memory and executed by aprocessor of a processing device. For example, at least portions of thefunctionality for information density processing and analysis asdisclosed herein are illustratively implemented in the form of softwarerunning on one or more processing devices.

It should again be emphasized that the above-described embodiments arepresented for purposes of illustration only. Many variations and otheralternative embodiments may be used. For example, the disclosedtechniques are applicable to a wide variety of other types ofinformation processing systems, designs, design personalization, etc.Also, the particular configurations of system and device elements andassociated processing operations illustratively shown in the drawingscan be varied in other embodiments. Moreover, the various assumptionsmade above in the course of describing the illustrative embodimentsshould also be viewed as exemplary rather than as requirements orlimitations of the disclosure. Numerous other alternative embodimentswithin the scope of the appended claims will be readily apparent tothose skilled in the art.

What is claimed is:
 1. An apparatus comprising: at least one processingdevice comprising a processor coupled to a memory; the at least oneprocessing device being configured to perform steps of: obtaining aninformation density matrix for an input visual stimulus, the informationdensity matrix characterizing information density of feature points inthe input visual stimulus; analyzing content of the input visualstimulus utilizing the information density matrix to identify one ormore objects in the input visual stimulus; determining emotionalresonance scores associated with each of the one or more objects in theinput visual stimulus, a given emotional resonance score for a given oneof the one or more objects characterizing an intensity and an emotionalvalence of an emotional effect of the given object on a user viewing theinput visual stimulus; and modifying a design of the input visualstimulus to adjust at least one of the emotional resonance scoresassociated with at least one of the one or more objects in the inputvisual stimulus.
 2. The apparatus of claim 1 wherein the input visualstimulus comprises an image file.
 3. The apparatus of claim 1 whereinanalyzing the content of the input visual stimulus to identify the oneor more objects comprises identifying objects in one or moreuser-specified areas of interest within the input visual stimulus. 4.The apparatus of claim 1 wherein determining the given emotionalresonance score for the given object comprises performing color-emotionanalysis to generate a color-emotion score characterizing the emotionaleffect of the given object on the user viewing the input visual stimulusbased at least in part on one or more colors in the given object.
 5. Theapparatus of claim 1 wherein determining the emotional resonance scoresfor the one or more objects comprises classifying stimuli type of theone or more objects, the stimuli type comprising at least one oftext-based stimuli, static image stimuli, and moving image stimuli. 6.The apparatus of claim 5 wherein the at least one processing device isfurther configured to perform the step of determining an overallemotional resonance score for the input visual stimulus as a weightedaverage of the emotional resonance scores determined for the one or moreobjects, wherein weights are assigned to the one or more objects basedat least in part on the classified stimuli type of the one or moreobjects and relative object size of the one or more objects.
 7. Theapparatus of claim 6 wherein determining the overall emotional resonancescore for the input visual stimulus is further based at least in part ona user-input emotional resonance score component for the user viewingthe input visual stimulus, the user-input emotional resonance scorecomponent characterizing external factors affecting emotion of the givenuser at least one of prior to and during viewing of the input visualstimulus.
 8. The apparatus of claim 6 wherein the overall emotionalresonance score for the input visual stimulus has a magnitudecharacterizing an overall intensity of an emotional effect of the inputvisual stimulus on the user and a sign characterizing an overallemotional valence of the emotional effect of the input visual stimuluson the user.
 9. The apparatus of claim 1 wherein the given objectcomprises text content, and wherein determining the given emotionalresonance score for the given object comprises: extracting the textcontent from the given object; and performing sentiment analysis for thetext content using a machine learning-based sentiment classifier. 10.The apparatus of claim 1 wherein the given object comprises one or morestatic images, and wherein determining the given emotional resonancescore for the given object comprises: utilizing a machine learning-basedsemantic segmentation network that identifies, for each of one or moreportions of the one or more static images, one of a set of semanticclasses; and generating the given emotional resonance score based atleast in part on the identified semantic classes for the one or morestatic images.
 11. The apparatus of claim 1 wherein the given objectcomprises one or more moving images, and wherein determining the givenemotional resonance score for the given object comprises: breaking downthe one or more moving images into two or more static images; generatingan emotional resonance score component for each of the two or morestatic images; and determining the given emotional resonance score forthe given object based at least in part on an average of the emotionalresonance score components for the two or more static images.
 12. Theapparatus of claim 1 wherein the at least one processing device isfurther configured to perform the step of generating a visualizationcomprising a grid of equally spaced bins overlaying the input visualstimulus, each of the bins being associated with a probability thatcontent within that bin will elicit an emotional response from the userviewing the input visual stimulus, the probability being determinedbased at least in part on the emotional resonance scores determined forthe one or more objects.
 13. The apparatus of claim 1 wherein modifyingthe design of the input visual stimulus to adjust at least one of theemotional resonance scores associated with at least one of the one ormore objects in the input visual stimulus comprises modifying said atleast one of the one or more objects to adjust its emotional resonancescore relative to the emotional resonance scores of other ones of theone or more objects in the input visual stimulus.
 14. The apparatus ofclaim 1 wherein modifying the design of the input visual stimulus toadjust at least one of the emotional resonance scores associated with atleast one of the one or more objects in the input visual stimuluscomprises at least one of: modifying information content of said atleast one of the one or more objects in the input visual stimulus; andmodifying a size of said at least one of the one or more objects in theinput visual stimulus.
 15. A computer program product comprising anon-transitory processor-readable storage medium having stored thereinprogram code of one or more software programs, wherein the program codewhen executed by at least one processing device causes the at least oneprocessing device to perform steps of: obtaining an information densitymatrix for an input visual stimulus, the information density matrixcharacterizing information density of feature points in the input visualstimulus; analyzing content of the input visual stimulus utilizing theinformation density matrix to identify one or more objects in the inputvisual stimulus; determining emotional resonance scores associated witheach of the one or more objects in the input visual stimulus, a givenemotional resonance score for a given one of the one or more objectscharacterizing an intensity and an emotional valence of an emotionaleffect of the given object on a user viewing the input visual stimulus;and modifying a design of the input visual stimulus to adjust at leastone of the emotional resonance scores associated with at least one ofthe one or more objects in the input visual stimulus.
 16. The computerprogram product of claim 15 wherein determining the given emotionalresonance score for the given object comprises performing color-emotionanalysis to generate a color-emotion score characterizing the emotionaleffect of the given object on the user viewing the input visual stimulusbased at least in part on one or more colors in the given object. 17.The computer program product of claim 15 wherein modifying the design ofthe input visual stimulus to adjust at least one of the emotionalresonance scores associated with at least one of the one or more objectsin the input visual stimulus comprises modifying said at least one ofthe one or more objects to adjust its emotional resonance score relativeto the emotional resonance scores of other ones of the one or moreobjects in the input visual stimulus.
 18. A method comprising steps of:obtaining an information density matrix for an input visual stimulus,the information density matrix characterizing information density offeature points in the input visual stimulus; analyzing content of theinput visual stimulus utilizing the information density matrix toidentify one or more objects in the input visual stimulus; determiningemotional resonance scores associated with each of the one or moreobjects in the input visual stimulus, a given emotional resonance scorefor a given one of the one or more objects characterizing an intensityand an emotional valence of an emotional effect of the given object on auser viewing the input visual stimulus; and modifying a design of theinput visual stimulus to adjust at least one of the emotional resonancescores associated with at least one of the one or more objects in theinput visual stimulus; wherein the method is performed by at least oneprocessing device comprising a processor coupled to a memory.
 19. Themethod of claim 18 wherein determining the given emotional resonancescore for the given object comprises performing color-emotion analysisto generate a color-emotion score characterizing the emotional effect ofthe given object on the user viewing the input visual stimulus based atleast in part on one or more colors in the given object.
 20. The methodof claim 18 wherein modifying the design of the input visual stimulus toadjust at least one of the emotional resonance scores associated with atleast one of the one or more objects in the input visual stimuluscomprises modifying said at least one of the one or more objects toadjust its emotional resonance score relative to the emotional resonancescores of other ones of the one or more objects in the input visualstimulus.